(a brief post - written before I left on vacation).

At some point, my engineering team at Microsoft decided that it would be cool if they had a “crusty old engineer” agent that reviewed their designs from a seasoned (if slightly jaded) systems engineering perspective. Not something that would find bugs per se, but something that would point out bad patterns and suggest better approaches. We built this for Amplifier, and it works quite well (and I encourage folks to find it in our repos and clone if they want).

Then one day I mentioned to one of the engineers, “you should build that, but for simplicity. I think y’all complicate things too much”. They said “you should do that”. So I did.

The process was, literally three prompts. One was “I want something like the COE but for simplicity. You can call it Cranky Old Sam”. Then it went off, did a bunch of research and came back with a design. Prompt 2 was me approving and answering a question about structure with “Make it just like COE, do whatever we did there”. Third prompt was “cool, make a private repo and share <another engineer on the team> to it”.

Then I forgot about it. Actually, forgot whether I had even finished it. A week later, a bunch of the team start to tell me that the COS is really great - partially from amusement value (I gather it’s a bit salty) but also that it’s useful. Just encoding that perspective, that simple is better, and to look through that particular lens, had value.

This was a really surprising bit of low hanging fruit for me. Particularly the absolutely minimal degree of effort. I can’t tell if that’s the model getting better, the harness getting better, or both (I suspect a bit of both), but it’s somewhat stunning that what would have been at best a lot of work for me to personally track down and review designs, think deeply about them, and then persuade engineers of my perspective…is just an automatic part of the environment now.

I don’t know what that means. I suspect more aspects of team and company culture will wind up getting baked into our agentic tools, of course. I think it will be hard to do this well - and the layers of metacognition, strategies for having strategies for thinking, and so on, will make it very complex. This might well be the kind of thing that turns into a secret sauce - teams with a particularly good set of interacting agents, principles and tools will just be more effective, and it might not always be obvious why.

Fascinating to see, though. So much of what we try to do is hard and fails at first. Having something like this work with so little effort that I forgot about it, is quite remarkable. Perhaps just a lucky bounce, or perhaps there are more of these out there…

Subscribe now

Curiosity is a hard thing to cultivate in yourself. It’s easy to settle into the familiar - efficient, even. Habits and patterns are natural things for all of us, and being curious, discontented, and trying new things is often hard.

But curiosity, and its cousin discontent, are central to being an engineer, scientist, or creator. You can’t get an answer to a question you haven’t asked yet, of course. Or can you? It seems pretty basic - the flow of the world is “ask question, get answer”, not “find answer, look for question”.

But it does sometimes work that way, if you’re open enough to it. Sometimes we get anomalies - things that don’t quite make sense or are unexpected. I like the story of the person who invented microwave ovens noticing that the chocolate bar in his pocket melted when he stood near a microwave antenna (also - yikes). Sometimes you find the answer first, and then questions suggest themselves (why did that happen? what else can I do with that?).

I think there’s a bit of a danger with LLMs, in that you rarely, if ever, get an answer without a question, and you have to be very, very careful about how you ask the questions you do ask. Sadly, LLMs are really only replying with the knowledge we’ve given them, and that knowledge is neither complete nor correct in its entirety. So the answer you get from an LLM is always suspect, sometimes more or less depending on the quality of real-world knowledge in that particular domain.

But that’s ok, I think. The world itself is fuzzy and confusing. Physics can’t ever really lie, but it can do very unexpected things. The history of quantum mechanics is a good example of this - weird answers coming out of observations leading us to propose weird questions and weirder theories. The ability to notice that weirdness and methodically interrogate it is critical to being a scientist. The ability to notice and ask hard questions about the irritations of the world is what makes a good engineer, or entrepreneur.

All of us now will have to learn to be skeptical questioners. LLMs are too powerful to ignore - they’re a great tool in many ways and getting better all the time. But like any advanced tool, they are dangerous if not well understood or misused. They’ll also happily waste time on the wrong question if you ask something badly, or ask for something you don’t quite want.

All of this is becoming more acute as agentic systems get more common. Everyone is working on getting more leverage out of our interactions with them - as we should be. Human attention is the scarce resource now, so we want AI to do lots of work per unit of our attention. But that work needs to be effective - it has to produce a good outcome. And that boils down to asking the right question at the start.

There’s no (good) answer without a (good) question.

(I am going on vacation for the next two Sundays. It’s remotely possible I’ll feel like writing something, but I likely won’t. Summer!)

Share

Subscribe now

I gave a talk this past week at Stanford - a friend asked me to come in on short notice, and we had a good, informal conversation with a room full of very curious students. It was great to see!

I got asked a question that I get asked often: how did I get the idea for Google Docs (Writely)? I’ve never felt the answer to it was that interesting. But I’ve realized over the years, that the person asking it is actually asking a different question: How do they come up with an idea like that?

And that’s an interesting question, because, to a first approximation, you don’t. And more importantly, when you do encounter an idea that might become something big and disruptive, it won’t look the way you think it will.

Everyone assumes there is a moment in innovation where the sky parts, an angelic choir sings, and you have an “aha!” moment. “I shall entirely reinvent the idea of applications and create cloud computing! Of course! How obvious!” or something like that.

But the real truth of these ideas, of course, is that they are always some mixture of dumb-sounding, hard to implement, non-obvious, or silly at the time. If you wind up saying “wouldn’t it be cool if X” and then “yeah but X would be so hard to do” - you’re there! Or if you say that, and someone else says “huh? What are you talking about? X doesn’t make sense at all”.

Try to imagine explaining Writely to someone 20 years ago before we had even mobile phones, and the internet/browser was mostly a new form of TV. “It’s a web page, but also it’s a document, you can edit it, but you’re not editing the page you are editing a document but actually it’s an application, not a web page and the document isn’t right there, it’s somewhere else and you can come back to this page but see many documents oh and also you are editing with other people at the same time”. Right now you understand what “a collaborative online document” is, because the idea has been around for a while. When it first emerged, it sounded like the above - a totally weird mess that didn’t make any real sense.

This isn’t unique to this idea. This is how it is with every idea, at first. Things are invisible, impossible, nonsensical, until…they aren’t. And then they become obvious almost instantly. And when people set out to find their own innovation, they’re always looking backwards at ideas like Writely, where it seems obvious, where it clearly works, where the hard problems have been solved so they are kind of invisible. That’s not ever the lived experience. The lived experience is that you have an idea that seems impossible, that people are indifferent to at best, possibly confused or even hostile, and it’s a royal pain on multiple levels to get it to happen.

This is the lens I try to look through in my work, and I’ve written about it many times (my apologies for those who have read this before - I thought we could use a break from technical topics). This is the “what if” mindset - if you see something hard, or impossible, or someone presents an idea to you that seems dumb, try to ask “what if” instead of “why not”. Why not is easy - every good idea will have a dozen stories you can tell about why it won’t work (try to find them in the current commentary for AI - it’s not hard). But the real skill if you want to make something that hasn’t existed before, is to look at that mess and think “what if it works? What does that look like? Can I fix those problems? What can I do to get through the thicket in front of me?”

I know there is a lot of confusion and hostility to AI right now. There are almost certainly some real problems that need to be solved, and it’s not at all clear what all of the impacts are - those are fair concerns. But they’re not reasons to reject it entirely - they are problems to understand, engage with, and work through. Everything new that we do, all of the progress we ever make, is hard at first. It’s always obvious later, good or bad. But in the moment, this moment, it’s confusing, hard, and not obvious at all.

Learn to dwell in that uncertainty and ask the “what if” questions - that’s the path to making your own dent in the world.

Share

Subscribe now

Like everyone else in the programming world, I’ve spent a bunch of time in the past few months writing “agentic” code. On the one hand, working with frontier models (and a nice budget) is incredibly productive. But it’s also, as most people are finding, exhausting. It’s easy to get something “almost done” but the model and then have to spend an unreasonable amount of time finishing it. I have a book writing project, for example, that is “done” but needs me to proofread a 70K word book that took the model a few hours to generate once I set the system up.

Why can’t we quite seem to get all the way to autonomy and completion? (at least for large tasks - for small ones, we often are, which is a hint). What’s the failure mode, and can we do something about it? I think there’s a deep pattern here that is worth digging into.

I’ve written before about the “impedance mismatch” between AI attention and human attention — the basic observation that AI scales (it’s just a matter of tokens, dollars, and GPUs) but human attention doesn’t (you only have so many hours in the day). This mismatch makes it easy for the AI to overwhelm you - - it never gets tired and there is always “more” of it. But there is something deeper going on that I didn’t fully capture, and it has to do with a strange fact about what LLMs actually are: LLMs don’t experience time.

I don’t mean that metaphorically, I mean it literally. When the model isn’t predicting tokens, it is perfectly quiescent in a way that has no human analog. There’s no waiting, no impatience, no sense that time is passing. It just stops existing, entirely, until the next token prediction is asked for. Between my prompts and its responses, nothing “happens” for the model — not because it’s fast, but because there is no experience of the interval at all1.

I said to someone once at a tech leaders conference, that all I wanted is for the LLM to be able to understand when it needed help or when it was ok. That person said “I think that’s the halting problem, but for LLMs”2. There’s some truth to that - LLMs (at least so far) don’t really have a great ability to self-predict. Humans don’t really either, but the fact that we can experience time at least points to some mechanism for it. If all you are is a disconnected pile of token predictions, I think it’s harder. (and I suspect the next phase of harness engineering is going to focus on topologies of agents that help with this).

I don’t think that time blindness is the whole answer to why it’s hard to get to completion and autonomy, but I think it’s characteristic of it. An LLM that could perfectly identify when it needs human help would have to fully model its own future failure states. It would have to have some good intuitive sense of how hard something would be for a human to do. if you’re not careful, the models will definitely give you work to do, and not be particularly sensitive to how long it takes. I think there’s something like that going in here, where the difference between “I need a simple answer from the human” and “I need the human to go do a multi-hour deployment task” is, basically, invisible to the LLM.

The consequence is that human attention in agentic work becomes “all or nothing.” If you can’t fully trust the “interrupt” signal - if you know the model will tell you it’s fine right up until the moment it isn’t - then you have to stay close. There are mitigations to this - we do things like “antagonistic review” where one LLM session reviews another, you can set up automated tests, break things into small pieces, be careful about setting clear goals before you begin, use the recipes technique we’ve used to get some determinism in where appropriate, etc. With care, you can reduce the amount of intervention (spent human attention) but it’s hard to reduce it to 0 (which is why it makes me think of the halting problem).

I don’t know what the real answer is here (when I ask LLMs to review my writing, they always say my endings are soft. Sorry! I’m exploring all of this too!) It might be the same as it is with humans - there’s no perfect answer, only an asymptotic one, and we need systems that we can learn to trust over time (which means they need to be able to learn from mistakes, have a sense of self (or at least of their behavior and failure modes) and maybe even a sense of time). I think this is another one of those areas, though, where our instincts fool us: they “seem” like “people” so we assume their “experience” (strong quotes - don’t “@” me) must be similar, and we are surprised when they behave in ways that show that it really isn’t.

Share

Subscribe now

For most of the computing history, software has worked by turning meaning into syntax. That’s all we had - we had to content ourselves with programs that are hand-written and deterministic, and at best, weak representations of the real world. I tend to think of these with the shorthand “syntactic” (by syntactic, I mean systems where the important work happens because a human has already converted the messy world into explicit rules, structures, schemas, and code paths). This is the best we could do - the real world is complex and varied, computers are, after all, only simple math, and it’s hard to express that complexity in that simple model. We invented all kinds of programming languages, database designs, protocols and standards that allowed us to capture the wildness of the world in fixed syntax that we could actually work with.

But of course, this is brittle. A simple computer program can only deal with exactly whatever the programmer considered when writing it. LLMs are stochastic - they don’t do the same thing twice - which is a drawback, but they are also “semantic”: they understand meaning and can work with ambiguity, context, and resemblance directly, instead of requiring all of it to be precompiled into syntax. It’s possible to build really incredible things with them, though that comes with some costs, like perfect predictability.

But there is a failure mode that is very common the more senior the engineer: a desire to “go back to” the syntactic and deterministic world. This can manifest in a lot of ways, but often it shows up as someone trying to wrap a lot of code around an LLM in a subconscious attempt to get away from that uncomfortable randomness, and back to the world of nice, deterministic programs. The instinct is understandable. Senior engineers have spent decades learning that reliable systems come from explicit control. So when an LLM feels slippery, the reflex is to add another layer of control around it.

But the problem with this is that those techniques really never worked well for the kinds of things we want to do with LLMs. The second you step out of the semantic world, you are now trying to build something like an expert system or a part of the AI by hand - it didn’t work then and it’s not going to work well now. If you find yourself thinking “just one more patch” to your controller or harness, you have probably fallen into this trap.

One analogy I think is useful is the transition between the artisan and mass-produced worlds. Both have their place and are useful, but aren’t particularly compatible with each other. Imagine trying to build an early factory line - it’s all working, using standard parts, going well, except there’s this one spot that is fiddly. Let’s bring in a craftsman to sit at that spot and hand make the troublesome part! That, of course, wouldn’t work at all - the better approach is to understand what the failure is, deeply, and do the novel engineering to get past it - new materials, machines, or techniques. The factory line is all or nothing - you can’t go “a bit” back to the artisan world.

The world of LLM coding is, I believe, the same. To build effective systems, you have to be very clear about how you are using, or not using, the LLM. In the mass-produced world, there are still artisans - for example, the patternmakers who make the molds and dies for the production line. Similarly, in the LLM world, code has a place, but it is mostly used for “meta cognitive” things like control, that are hard or awkward for a model to do inside the inference API directly. Code should increasingly move from doing the cognitive work to shaping the conditions under which the cognitive work happens - that’s what “meta-cognitive” means.

We are in an era of inventing this new kind of engineering. It’s always tempting, when a new paradigm appears, to imitate the old one. And for a time, that can be a useful way to get our bearings. But ultimately, the real power and advances of the new system will come from a native understanding of it that builds to the new paradigm in the ways it needs. The real engineering discipline here is not “how do we make LLMs behave like old programs?” It is learning where syntax still belongs, where semantics should be allowed to operate, and how to build the scaffolding that lets both do their proper work.

That boundary is where the next generation of software engineering is being invented.

Subscribe now

Share

I mentioned last week that I have been working on what has variously been called “agentic coding”, “AI coding” and now “harness engineering”. Someone mentioned that perhaps not everyone understood what a harness is, or why they matter, and suggested I write about it this week. Why not!

When GPT-4 was first shown to me at Microsoft almost 4 years ago, one of the first things I thought was “well, there are a lot of people working on building models, I probably can’t help with that, but I do know how to build software, so I’ll focus on building WITH models instead”. The first project my team and I worked on in this direction was what was then called an “orchestrator” - ours was (and still is) called Semantic Kernel.1

The kernel was very simple - just some bits of code that could “orchestrate” calls to an AI model. How does that work? Well - most people encounter AI through the chat interface - you talk to it, wait for a response, etc. There’s another interface, an API, that you can call directly. It’s similar - you send in a prompt as text - but you can do a bit more with it. You can control things like the “temperature” (how “low probability” the responses are), stream token-by-token (instead of waiting for either upload or download) and so on. So with an orchestrator, you can do simple automation, like running a loop that calls the API multiple times. You can take the response of one request and use it in code to make decisions about what the next request might be. One early thing we did was called “context query”, where the first request was a set of questions and an instruction to return them in XML format, with each categorized (“fact”, “unknown”, “needs web lookup” etc) and then a second pass would either generate the answer directly, if known, or invoke a tool like search if needed. It cut down on hallucinations.

Tool use like search is another thing that harnesses can do. Modern harnesses allow models to make very sophisticated use of tools. The one I’m working on lets a model, or agent, create a “subagent” and hand tasks off to it, as though it were a separate worker. Or it can create a whole “container” that isolates work into a new virtual environment.

Harnesses allow models to select tools and agents based on the task at hand, which gives much better results than “just asking”. We use a technique called “recipes” that are workflows based on a format called yaml - these orchestrate calls to the model into long, complex tasks. This is what I used to build the Word clone I mentioned. This past week I built a large recipe, that I call a “foundry” that is capable of writing books in my voice. It uses a huge collection of tools - custom models to judge the writing, linguistic analysis tools, and some custom editing workflows that interact with me.

Harnesses can also get around, at least partially, one of the limits of AI models: memory. Even though an AI model seems like a person sometimes, it doesn’t really have the same kind of mind as a person does - in particular, it can’t form new memories. All new information has to be passed in, each time. One of the main uses of an agentic harness is to manage and orchestrate an external memory store. Keeping the “context” that is passed to the model in good shape is a hard problem with a number of different approaches that agentic harnesses can implement.

One of the challenges of working on a harness is that the models will often overtake some of the capabilities in the harness - this has been true of memory, some tool use, and other kinds of reasoning and metacognition. So it makes harness development somewhat of a moving target. My team has found that we have always been able to get more out of a model with a harness, than without one, though - there’s never been a time when we couldn’t develop some new leverage. The problems we can reach for just keep getting bigger.

And, even more interestingly, now that the models themselves understand code well, they can help with their own harness development. The harness we work on (Microsoft Amplifier, open source on GitHub) has two different experts (or tools, depending on how you look at them) that help. One is called “session analyst”, which is a tool it can use to look at other harness sessions and understand how they work (this is actually a fairly hard thing to do well - the amount of text in a large session can easily be more than the model itself can handle in one go, so it can’t just read it all in - even just reading enough to understand what to read can overwhelm. It’s a little like reading out of the corner of your eye - has to be done carefully). The other is a “foundation expert” that understands how the system itself is put together (a third, “Crusty Old Engineer” is my absolute favorite - a cynical engineer agent that gives really good design advice). Combined, these can often look at a new capability, another codebase, or even a research paper, and work together to add new capabilities to the harness itself. Others have called this “gene transfer” and it works surprisingly well.

And of course, all of this compounds - as the harness builds tools, it gets better at building the next ones. And the models get smarter, so they get better at writing code and using the tools, and the harness can take advantage of that. Over time, we’ve gone from very simple tools like Context Query, to the current ones that are building programs, books2, and more complex systems and artifacts over days or even weeks, with little or no supervision.

So…that’s what a harness is. When you hear “AI”, it’s important to think about the fact that it’s not one single thing. There are models, that do the core work of thinking, but then there are tools, data connectors, memory systems, and finally harnesses that orchestrate all of it into even more effective systems. Each piece contributes something to the overall picture.

Subscribe now

Share

There’s a very cool exhibit in the San Francisco Exploratorium, called “Machine with Concrete”. It’s a long series of reduction gears with a running motor at one end, and a block of concrete at the other. One end is turning furiously, and the reduction in gearing means that the end will only turn once every trillion years or so (the web page says billion but the exhibit says trillion). It’s a fascinating display of entropy and thermodynamics, but I’ve also always seen it as a good illustration of the idea that activity (or motion) isn’t the same as progress.

I’m a very creative person. OK, actually I have ADHD so I have a LOT of ideas, all the time (it’s practically impossible for me to clean up my home workshop because I will find some distracting project within 15 minutes of starting, every time). AI has been really great for me, at least in one way: it let’s me bring my ideas into reality very quickly.

My team at Microsoft has been building a very powerful agentic harness called Amplifier for the past year or so. It has a vast number of tools - architecture experts, reflective “session analyzers”, a “dev foundry” that I built that stamps out bespoke “dev machines” that can work on tasks for days or weeks on their own, and my favorite, the “Crusty Old Engineer” who is painful but always gives super valuable feedback.

This framework makes it very easy to just start something (and Microsoft gives us a generous token budget - this is expensive!) So my ADHD brain is in heaven - I have many ideas every day and it’s very easy to start a project to go investigate them. The harness knows me, knows its tools, has access to my whole enviroment, and is quite effective. A few months ago I had a GitHub user with no repos at all. Now I have over a hundred. The machine is spinning!

But…not much of that is fully shipped software. Not because it’s impossible, or even hard, or that the quality isn’t high. The reality is a bit more subtle: shipping, going all the way to robustly done, still involves some things that aren’t very well “agent shaped” - little bits of fit and finish, context understanding, making sure the final product is really what I think it is, etc. Some of this can be mitigated - and I think all of it eventually will be - but right now, the “last mile” of releasing something is still too manual.

The machine of my creativity combined with AI spins. The concrete of shipping sits there, unmoved.

There are two major areas that my team and I are starting to focus on, in our work with these tools. This is the first: that finishing something should be as easy as starting it. I think that’s possible, and I think largely we have all been so enamored of how easy it is to start things, that we just haven’t focused on it yet. The second area is working together - both many agents working on a problem at once, without causing a mess (so the “wall clock” time to finish it is faster) and people working together in teams. My team of 12 has over 500 projects we’ve built - it’s hard even to understand with the help of AI.

Why am I telling you this? Sounds like a failure - lots of heat and motion but no output. I don’t think it is. I think it’s a necessary step along the road - before the current wave of models, we had the idea of a harness, and some of the pieces, but it wasn’t very effective, and we spent our time on that problem. Then we had effective models so we could dive deeply into making the harness smarter, self-improving, and more effective. Now that we largely understand that, we have uncovered the next problem - the last mile. All technologies are like this - one problem or bottleneck at a time. Solving one reveals the next.

I know there is a lot of backlash and suspicion these days about AI. My experience as a technologist leads me to believe this is misplaced. Yes, it’s being used in dumb ways right now, mostly to rent-seek and reduce costs. This is largely because those use cases aren’t complex either to understand or implement. But the work I’ve done with Amplifier has convinced me that there are huge gains waiting for all of us, once we get through the building phase. Last month I used my tools to clone Word - I’ve already seen one other person do this, and it’s not even particularly interesting now. This week I’ve been working on writing novels - they’re not good yet but they’re getting better. I’ve built two different large, complex tools for someone else, just because it was easy to help them - one for internal use and one for a school in Japan, to help interview students and help them shape ideas for AI.

If you’ve read my writing for any amount of time, you know that my constant theme is that pessimism usually isn’t very valuable, that it’s better to ask “what if?” than “why not?”. Every new mode of being, every new tool, every new food or new idea we’ve ever had, was “wrong” before it was “right”. There are problems with AI still, but we are gradually solving them, and it’s already clear to me that there is huge value every time we solve the next one. Things that were unthinkable even a few months ago are routine or even boring now. The gears are all turning. The block will move, eventually.

Subscribe now

Share

I think there’s an interesting connection between the obesity epidemic and AI. Bear with me on this.

Historically, calories have been hard to come by. Our physiology is adapted for feast or famine: when we see high-calorie foods, like sugars and fats, we eat as much of them as we can. In our prehistoric environment, that was an adaptive behavior: those foods were rare, the calories were essentially always valuable, so there’s no point in limiting consumption.

Now, though, the world has changed. Calories are cheap, and we make artificial foods that deliberately stimulate the signals that tell us to eat with abandon, so we do. Hence: the obesity epidemic. It’s easy to get calories, hard to be disciplined, easy to get fat.

What does that have to do with AI? Well, historically, it was hard to do cognition (and more recently, get any kind of intellectual work done). I wouldn’t quite say we’ve adapted to it in the same way, the timescales are different, but it’s very similar: thinking used to be hard, and expensive.

And now, it’s not. In some domains, like software, it’s very easy to get a LOT of work done. In others, it’s easy, but the work is somewhat like junk food: writing is like that. Easy to get words, hard to get good ones that sound human (I actually wrote this entire article originally with a system I’m working on, got halfway through editing it, decided this analogy was more interesting, and tossed the whole thing, to write by hand, which I am doing now).

In both realms, the real value comes from choice: having taste in what is healthy to eat, good for you, actually good to consume, is much more important than the raw necessity of just getting access to food (assuming you do have access to food - sadly not everyone does, even in modern day America). In AI, it’s really easy to ask for something to be done, and to get a partial result that needs your attention - and if you do this badly enough, you can spend more time and attention fixing the output than you would have to begin with. Or you can get mentally “flabby” and create and consume low-grade output because it’s easy, in the same way that a bag of chips is1.

Like it or not, our mental environment is changing in the same way that our physical one did (and in some new ways). It’s absolutely critical to develop new skills to match the new landscape - as with all evolving systems, previously adaptive behaviors can easily become maladaptive when circumstances change. Whatever else AI is, it is a massive change to our environment.

Subscribe now

Share

There is a pattern in evolutionary theory, called ‘punctuated equilibrium’. Roughly speaking, the evolutionary record seems to show long-ish periods of relative stability, and then rapid periods of change. These typically happen when some major environmental change happens - genes that were previously harmless or even a little bit maladaptive are suddenly very adaptive and thrive. In humans, a few examples of this are the rapid rise in lactose tolerance in Europe as some humans domesticated cattle, or the gene that, if you have two copies, causes sickle-cell anemia, but if you have only 1, confers some protection from malaria. There’s also a famous example of moths in England becoming darker as coal soot from the industrial revolution darkened the trees they hid on.

One thing I’ve noticed lately, is that folks with some “neurodivergent” traits - mostly ADHD1 - seem to be really thriving with the world of Agentic coding and AI. It’s not totally clear why this is - it might be that we just like novelty (there’s a theory that ADHD traits are adaptive to hunter-gatherer environments, where novelty seeking and rapid attention shifting are survival skills). It might also just be that the ADHD personality is well suited to skimming back and forth across a number of working AI sessions. It’s interesting that the traits that used to be valuable, weren’t for a while (it’s hard to have ADHD in a classroom) and now they are again. That happens in genetics too - things lie dormant, as long as they aren’t too maladaptive.

Some of this is also happening with software itself, and sometimes in reverse: small SaaS products that used to be at least fairly successful are now much easier to build and use as AI prompts. Someone I know who entirely non-technical has built a commerce site for themselves, highly technical and specialized, with no help other than Claude code. I’ve been helping my wife do research on non-profit grants that are in the public domain. Both of these would have been very expensive before. Now they are almost afterthoughts. Companies that provided these services aren’t competitive now. Sometimes the rapid shifts cause harm instead of benefit.

In fact, that’s not quite an iron law, but probably close to it: if the configuration of the environment changes radically, there’s something close to a zero-sum game that happens. Some traits will make their owners more successful, and some will make them less.

The thing to notice now, in the early stages, are the small things that are suddenly thriving. It’s easier to see the big things that are struggling, and there is plenty of anxiety about that. But just with earlier major technical shifts, most of the interesting stuff will be the new capabilities. Sometimes these will be invented from scratch for the new world, but often they will have been there in the background all along.

It’s super easy to write these off. You’ll hear about something that is suddenly doing well and think, “well, we always had that”. eBay was just an auction. Google was just a search tool. The web was “just” like a newspaper. Or something.

But the way the world will reorder itself is the way it always has: selection and adaption, otherwise known as evolution. The things that are now maladaptive will fade, whether those are personality traits that are currently rewarded, software approaches, corporate cultures or whole businesses. And the things that are adaptive will emerge and proliferate. No one will consciously choose this, or can control it, it’s just how selection works.

One thing I think we will see: the way companies are organized and run, with hierarchy, process, meetings, etc, works but is mostly “the best we have come up with so far”. In the AI world, small teams are highly effective, and big teams struggle with communication overhead. AI has changed many of the underlying assumptions of these organizations - it’s far easier now to survey, explain, rewrite, build compliance, communicate, track, etc. Many of the manual mechanisms we’ve built are likely now accessible to automation. The organizations that figure this out (e.g. mutate) first, will thrive. Those patterns will outcompete slower, older patterns.

If you combine this with some of the other things I’ve written, such as the idea that human attention is now the primary constrained resource and optimizing for it is the primary organizing principle now, you will have a fairly good set of tools to help navigate the rapid change.

The meteor has hit. Be a small mammal, not a dinosaur!

I have ADHD. I’m super impatient. I want things quickly, and I tend to not do a lot of planning before I start projects - I like to just dive in.

AI is like an eager puppy (an expensive one!) - it’ll do whatever you want. You can prompt it to push back, if you think to, and it won’t do obviously bad things unless you work hard to trick it, but mostly, it doesn’t have “agency” or context and just follows instructions.

That’s kind of bad if you have access to powerful tools. It’s very easy to kick off a large, long-running project with an Agentic harness, even if it has a poorly defined goal. The AI will happily pursue that goal, and you’re stuck with something that’s not complete, workable, or well thought out.

I’ve found in my own projects that if I take the time up front to think through the problem, I get much better results. And more than that, I have to think through, and express, my intent. If the model understands that clearly, and if I refrain from being overly prescriptive on the how, and mostly focus on the what, I get much better results.

There’s a lot of talk about the “deflationary era of tech debt”, and it’s certainly true that these tools let you retire issues in a cost-effective way that wouldn’t have been possible even a few months ago. But failing this “modern marshmallow test” is giving rise to a new and different kind of tech debt: large, complex projects that are functionally and syntactically correct, even well organized (as opposed to what we usually think of as tech debt), but poorly thought out and designed, and as a result, very hard to work with and integrate.

This kind of debt is subtle. The models do usually manage to build something workable, or at least sort of workable. You find yourself chasing endless bugs because somehow, the model can’t understand what it’s built well enough to debug it, or to build it properly. Sometimes the model can’t see enough of the problem at once to understand it, sometimes the problem just has enough contradictions or under-defined areas to cause problems.

The best cure for this is patience and using the models themselves to help you take your time, pressure test the idea, and work out a plan. This goes for non-technical tasks too - often the models will answer questions without really understanding what’s being asked, and it’s worth taking the time to make sure they do, particularly with more complex problems.

If you’ve tried to do something with AI and failed, it could be that you haven’t asked the right questions in the right way. Have it ask you for clarity and have a conversation before trying to solve the problem.

Share

Subscribe now

The internet taught all of us to expect that almost everything - devices, businesses, software of any kind - is connected. If you can’t find a business online, it might as well not exist now. If I gave you a laptop or an iPhone that can’t connect to the internet, it might as well be broken.

That came with some costs though. Everything being connected, meant we were connected too - everyone in one big room, shouting all the time. In the early days of what used to be called “user generated content”, people didn’t have good habits for this (and, to be honest, mostly still don’t), and got overwhelmed trying to keep up. I remember seeing a blog post titled “The Internet is stronger than you” that made the point that, at some point, you have to give up - you can’t consume it all. There is so much content now, this seems obvious, but there was really a time when people felt some stress of trying to “read it all”.

To me, this rhymes with something that’s happening with AI now (hence the title). In many directions, we see examples of people being exhausted or overwhelmed by AI - agentic coders are working themselves to exhaustion and are generally overwhelmed (Steve Yegge talked about AI vampires, which I liked). People are being confused by AI and having mental health issues, “falling in love” with an AI, and so on.

Just as the internet taught us to expect connectivity everywhere, AI is teaching us to expect conversation and responsiveness. We are entering an era where we can ask the computer, in plain language, to do what we want, and it will (largely) do it. That expectation will spread through everything we build, and in many ways, will be excellent.

But just like the internet, it comes with a cost. AI is stronger than you - it can talk forever. It will never get tired. And if you ask it to do something, it will do that thing, to the best of its ability. But…often, if you’re not careful, the thing you ask it to do will not be quite right. And, being not quite right, will generate some work for you.

This is what’s happening with the agentic coders, I think. I see it with myself, at least. I can have an idea, and fairly quickly tell my coding harness to start working on it. That harness, and the AI powering it, will do a gigantic amount of work in the direction I tell it to. But…it won’t really finish the job. I always have to watch it, make course corrections, and get it the “last 20% of the way”.

I’ve been building “dev machines” that will run for days and build very large projects, on their own. Last week I ran 5 of them. Currently 3 are sitting idle, and have been, for days, because they’ve finished the work I gave them, and now it’s my turn to evaluate. The AI gave me work!

The AI is stronger than you. You can talk to it, ask it for help, have it do work. But it will ALWAYS respond. It never gets tired, and it will happily make work for you to do, unwittingly.

I think there are amazing things to be built with this technology, just as there were with the internet. But there are also dangerous and unhealthy patterns to understand and build habits for. This is one of them. It’s very easy to be pulled into an “ambition loop” where you try to get more and more done with the AI but wind up with more and more work for yourself. I think this is the root of the exhaustion that many agentic coders are feeling now - it’s so easy to start new conversations that then have a carrying cost down the road.

We all had to build new mental hygiene for the internet. Some better than others (and I strongly recommend getting off any feed-based social media, as a best practice). We are going to have to do the same for AI, and for many of the same reasons. AI is stronger than you. Control your inputs, and when you ask for something, be mindful of what you are being asked for in return.

Share

Subscribe now

I had the privilege of being on the founding team of what became Google Docs. It was an interesting experience in many ways, but one of them was experiencing firsthand the dissonance between people telling me it was a terrible idea (not just Writely, but the cloud overall), and the half million or so people who were using and loving it.

I’ve been hearing AI is a bubble, doesn’t work, vibe coding is terrible, etc. And yet, in the past 20 or so days, a piece of scaffolding that I built, that I call a “dev machine” has been running independently, building out a pretty complete online copy of Word - collab, import, all the editing features, etc. It has two test pipelines (one at the code level, one for rendering fidelity), a CI deployment pipeline, does refactorings, performance tests, and so on. I doubt I will make it fully public, but I use it to edit markdown - it’s wonderful for that, and adding that feature was as simple as asking for it to be added to the backlog.

Dissonance, again.

Why do we get this dissonance? I think there are a few reasons, but they more or less boil down to emotion. When we see something disruptive, that challenges our world view, we are personally challenged - our place in the world is threatened, and our ego, about how well we understand the world, gets hurt. We react emotionally - either with humility and learning (hey, maybe I didn’t understand something about the world!) or rejection (I can’t be wrong, so New Thing must be).

So then we want to tell a story about the emotion. People are often confused, and think the story comes first, but it doesn’t. We justify the emotion with the story, not the other way around. And most things are complex - they have many dimensions to them, some good, some bad, some accurate, some not. So it’s very easy to pick one or two dimensions, focus on them, and find a “why not” story. Or sometimes we just have selection bias - “AI coded badly sometimes, therefore it codes bad always”, for example.

“Why not” stories are stories where our emotion is negative, and we are trying to justify it. “What if” stories are where we are more positive in our emotion and are looking for “what happens if” the new thing is true.

With software, the “why not” stories are mainly that a) the code is bad/insecure/something and b) programmers will all be out of work. Both are a little true: some AI code is bad, particularly if you use the tools badly (don’t test, etc). Some programming jobs are going to go away, no doubt - the nature of software just radically changed.

But…neither capture the full story. We are going to have a LOT more software - we like software, it’s computers doing things for us. It’ll probably be more like media - everyone can be a creator, there are new platforms that support common needs (for media, distribution, moderation, scale. For software, safety, scale, maybe distribution, probably some new things). There will be new things to do with software - lots of folks who are part of big teams now may be independent, and happier (just like it was hard to make a living as what we now call an influencer back in the pre-internet days - and now the world is much richer for the fact that many people can bring their creativity to the space of media).

I’ve been really astounded at what these dev machines can do. I have 5 of them running now - but 3 of them are paused, waiting for me to test them and give feedback, done with their tasks. I don’t feel useless, to the contrary, I have a lot more to do now, and I have to be very careful where I spend my time and attention, because I get overwhelmed if I’m not careful, with all of the things I can build.

No one really has the answer to AI, not me, not any pundit. It’s complex, fast moving, multidimensional. We all want ways to navigate it. The best way is to tell the “what if” stories when you can. Try to observe yourself - when you tell a “why not” story, try to separate from your emotion and look at it as dispassionately as you can (or even talk it through with AI!)

I am excited, exhausted, and yes, somewhat worried about all of this. All kinds of unpredictable change is coming. But every day, I remind myself to think “what if” and not “why not” if I can.

Subscribe now

Share

In the past 30 days, I’ve written (well, caused to be written) more lines of code, than in the entire rest of my career (I measured - currently closing in on 1M lines of code). And it’s not just “TikTok for programmers” as a skeptical friend called it. One of the things I’ve built is something I call the “dev foundry” - a process, and program, for building other programs, whose jobs are to build and run complex development processes. One of these “dev machines” spent the last 9 days building a high fidelity clone of Word, but with web technologies. One built “openClaw but for enterprise m365”. One built a security filter product for agents that is also meant for enterprise. Each of these runs largely independently - including design, testing, and pushing to staging.

This is an extreme form of what is usually known as an “agentic harness” - software that makes the underlying model more effective. Models on their own can’t run for weeks at a time, but the harness (‘dev machine’) can - it can manage memory (state), start and stop process, make tool calls, etc. And the recent wave of models is good enough that they can help build the harnesses - much of what I do is improving the tool itself, as I find errors and gaps, or have new ideas. At this point I am largely limited by my own ability to pay attention and verify the output (remember that - we’ll come back to it in a minute).

So…who cares? Programmers are happy, the field of programming is going to be disrupted in some way that no one quite understands yet1. If you’re not a programmer, why be bothered?

Well…”code goes first”, right?2 Much of the world is digital now, or accessible to software. Is that “money” in your “bank” “account”, or entries in a database? I could go on - look around though and most of what you see is impacted by software somehow - your clothes are designed, manufactured, shipped, advertised and sold with it, your food is grown with it, your rent is paid with it…the world is a largely digital place now. Beyond that, much of what we do in the modern world is think - we do accounting, marketing, medicine, investment, law…a great deal of economic activity is thinking, thinking is another word for software, and it’s all digital now.

So, what is happening for programmers is going to start spreading into other realms soon. It’s easy to look at just the base models and see errors and limitations (and it’s surprising to me how many people who are critical have either never seriously tried to use these tools or use the free (and dumber) versions. It’s a bit like saying “I chatted with a high school student who thought they might want to go into medicine someday. I wasn’t impressed. I’ll stick with the witch doctor”). Building a domain-specific harness for a model can improve performance dramatically (one of my side projects is a foundry for domain experts that does continual “adult education” for itself).

So, what happens when you can build, or do, or think about anything? The cost of “capability” is collapsing. In a world where anyone can do “anything”, what starts to matter more is something like taste, and judgement: what you choose to do is more important now than that you can do it. Coding skills that I had 30 years ago, that were rare and valuable, are more or less free now. But my taste as a designer and architect is important - I often build better things, or more successful ones, than others, because I can make better choices.

I’ve thought for a while now that we need to collectively become better if we are going to survive and thrive with AI. All technology can be used for good or evil purposes, and this is a very powerful technology. That’s not novel. What I think is a bit more novel is that we also have to be better individually. We have to be curious, we have to choose well, we have to be thoughtful about how we personally use these tools. We can create AI slop that distracts and annoys others and mostly doesn’t add value. Or we can be thoughtful and use this new form of thought to better educate, help, and take care of each other.

The models are brilliant, and capable, but also dumb. They’re happy to do what we ask, even if it’s pointless (mostly they try not to do anything that’s actually harmful). As capable as they are now, they are only going to get better. Choice, and taste, is only going to matter more.

Code goes first, but it’s not the only thing. We all have to learn to choose well.

Subscribe now

Share

When I was in college (yeah, back before there was fire, we still had college) the Mac Classic was a new thing, and the nursing school across from my dorm had a whole room full of them, hooked up to dot matrix printers (ok sorry, Daisy Wheels sounded better in the title). I couldn’t type very well, so the fact that there was a word processor that I could correct mistakes on and then print, was AMAZING. I did all of my papers that way.

Incredibly, there was pushback at the time from some teachers, about the printing itself. Those printers weren’t nearly as nice as what we have now, but they were as good or better than typewriters of the day. But…they were different. And some teachers didn’t like that. They felt that we were “missing out” on something vital by not typing on typewriters (mostly what I was missing out on was either wasted time retyping or paying someone else to do it). Fortunately, sanity prevailed and computer edited and printed papers were accepted. Hard to imagine that’s an issue now, isn’t it?

Except…the above paragraph almost works if you substitute AI. Not perfectly (more on that below) but to a degree. In some ways, the work being done is better and faster than doing it manually (in some domains). In writing, it’s detectable - and once someone detects it, there is a tendency to reject it (remember the lesson of “why not vs what if” - people start with emotions and find reasons later. If someone already has an emotion about change - and most emotions about change are negative - they will find a way to object to it). But often…it’s fine. Better in some cases. Easier. I don’t know what the right use of AI is in school, but it’s not zero - just like the use of digital word processors and printers wasn’t zero, even though they hadn’t existed for the hundred years before.

The one challenge is what I call the “monkey with a machine gun” (after one of my favorite internet clips where someone, yes, gives a live machinegun to a monkey and … comes to regret it). There was a problem with the early word processors: taste. If you didn’t have good taste, you wound up being very excited by the idea that you could have OUTLINE and SHADOW and WINGDINGS and all kinds of crap in your (now ugly) document.

We get that now with AI. In coding, it’s particularly acute but it’s happening everywhere. If you have good intention, access to a model, and poor judgment or taste about what is valuable, you can create a lot of damage (slop) quickly. People who used to be mediocre programmers, but largely ineffective and slow, can now cause distraction or damage rapidly.

Which doesn’t mean to not have it or use it. What it does mean is that taste, skill, experience, judgement - all those nice human characteristics - are now as important as ever, or maybe more so. Being careless with a screwdriver or a hammer is one thing, with a chainsaw or a bulldozer, is entirely another.

People debate whether technologies are good or bad, but that’s almost certainly overly simplistic in most cases. There can be a bias, for sure - some things are more or less inherently dangerous and hard to use - but in general, at least at the broad scale, they tend to be dependent on how they’re used. I think AI broadly has the potential to be very good - I’ve written more code in the last 30 days than the entire rest of my career, which is astounding - but it isn’t free, or automatic. We still have to bring our best selves to using it, and we still have to try to be honest and open about the change to manage it successfully.

Share

Subscribe now

In 1997, Clayton Christensen wrote a book called “The Innovator’s Dilemma” where he described a common, and difficult business pattern of that name. The pattern happens when an established company is disrupted by a new entrant to the market that takes a particular shape that is hard to compete with. (It ought to be called the “incumbent’s dilemma” I think!)

Incumbents are large, established companies that have figured out how to serve a particular market. Incumbents can’t really address everyone in a market, usually - there are often use cases that either don’t fit with the main business or are too small or otherwise not economic enough to be worth going after. The incumbent has to make rational decisions to invest where there is the greatest return.

Sometimes, a technical shift lets a new entrant do something that is cheaper, usually “worse” in some sense, but solves a problem for some use case or part of the market that the incumbent can’t or won’t get to. If you are selling something for $100 that is carefully crafted over time, does a wide variety of jobs, and is complex, it’s hard to then sell something simple, incomplete and cheap for $10 to a customer who wouldn’t have bought the $100 thing.

This is fine except that the new entrant can grow, over time. This is where the dilemma is: in the near term, the new solution is “worse”, and it’s not economic for the incumbent to compete directly with it - they’d have to voluntarily destroy their own business, which is very hard to do structurally. But eventually, the new entrant can replace them - the simple, broken feature gets better and more complete, and every day the new entrant picks off a few more of the incumbent’s customers. Best case is the incumbent retreats to a core market where the new entrant isn’t effective, worst case is that this is a path to taking over the incumbent’s role entirely1.

AI perfectly fits the pattern of disruptor for almost any cognitive industry. It’s cheaper and simpler than most solutions, so some customers (old and new) will use it because it’s accessible. But it’s “incomplete” and “inferior” relative to the existing solution, so it’s hard to sell it to established customers. And every day it’s getting better and picking off more use cases.

The challenge now though is that this isn’t one particular company trying to disrupt another - it’s a secular change disrupting everyone. This is much harder to deal with. A direct competitor at least has their own economic incentives and vulnerabilities, and you can build a responsive strategy for that. AI is just…technology. And often it’s doing the innovator’s job: providing a cheaper, easier, “worse” solution to markets that didn’t have a solution at all.

This is happening in markets that aren’t used to this kind of competition - like medicine, legal, information work, etc. The software and tech companies are used to this - it happens a lot in that industry.

I don’t know what the end state of this is. Unfortunately, this kind of shift often has a feeling of inevitability - someone finds a better way to do something, and natural selection takes over. It’s hard to resist, and hard to push inefficiencies back into an economic system, and, at a large scale, it’s not clear that we want to. Wealth comes from productivity, and productivity comes from finding and taking advantage of better ways to do things. Uncomfortable as it is, disruption is part of what makes the world better.

Unfortunately, we all have innovator’s dilemma now.

Share

Subscribe now

Ok, this one is about AI too, but you probably should read it.

As you know, I spend essentially all of my time these days thinking about this, and working with it. And I talk to a lot of people about it, too. I see the usual bifurcation - some folks are happy and embracing it, and some hate it and want nothing to do with it. We’d expect that - that’s the usual “why not/what if” dynamic from anything disruptive, and, since this is VERY disruptive, that effect is stronger.

What’s really interesting though, is that there are folks who are absolutely “red pilled”: getting a ton done, super engaged, often super tired, but overall very effective. I had a meeting today with someone I turned onto agentic coding a few weeks ago - he excitedly told me about turning Claude code lose on his marketing and sales, rebuilt a much better site, kicked off a bunch of sales processes he wouldn’t have been able to start, and more.

So, some people are getting it, and some aren’t. Why? Some of it is attitude, but not all, or even most. It’s something about the shape of their work. I suspect this is actually the key thing, and I want to try to explain what I mean.

The people with the tailwind have found surfaces that agents can grab onto. They often have things they can iterate and experiment on. They have data that the model can get to easily. Their workflows decompose into discrete observable steps. Their documentation is structured well enough that a machine can navigate it. Their interfaces are clean enough that something other than a human can drive them.

And there’s an aspect of “recursion” to what successful folks are doing. They often stop and look at their own process and ask what is or isn’t working, and feed that back into the model. The folks getting traction are finding ways to shape the work so agents can help and are looking to increase leverage (as we discussed last week). The folks running into headwinds are doing their jobs the same way they always did, maybe with a chatbot open in another tab. Incremental, not compounding.

One good analogy to this is shipping containers, believe it or not. When Malcom McLean standardized the metal box in the 1950s, it looked like a logistics story. A box is a box. But what the container actually did was standardize the interface between production and transport so completely that it restructured global trade. The revolution wasn’t the ship — it was what the ship could now assume about the cargo. That assumption, that everything was in standard boxes, meant that the work could be automated - instead of humans manually carrying cargo on and off ships, we could build automated shipyards, and cranes, and trucks and train cars that all handled that standard container.

Once the assumption was baked in everywhere, everything compounded. The leverage came from the assumption, not the container.

The interesting thing here is that, to some degree, we’ve tried to do that with software, but it’s much harder. Containers are easy - literally a physical shape. Software, intention, cognition - comes in many shapes. It’s been much harder to find standards - we do things like build database schemas or standards like HTML so that, in some domain, computers have a chance of doing standard, scaled work.

That’s starting to change with agents. Any time content is “plumbed in” so that the agent can see it, they can be effective. Having a system “shaped” for agents the way containers are “shaped” for shipping makes all of it more effective - models do well with small, composable, well-defined pieces, and they don’t mind doing “toil” to have them work. A model is perfectly happy with html and ftp to do a presentation, but a human wouldn’t be.

Once the idea of containers was in the world, there was a powerful motivation for everyone to adapt to that system - it was much more efficient, and it took over pretty quickly. The same is going to happen with software and all “intellectual worker” tasks. Having a program, data source, or process that models can’t access is going to be more or less like shipping something outside of a container - maybe possible, far more expensive, and hard to convince everyone to do.

So the folks that are “getting it” see this new world and are able to shift their work to it. They’re happy - the new tools make them more effective, and often help them do something that before wasn’t possible at all.

Here’s the part that makes me genuinely uneasy about this moment though: the people who don’t see this, don’t know they’re being left behind. That’s the thing that doesn’t show up anywhere. Their metrics look fine. They’re shipping their projects, hitting their numbers, doing their jobs. The gap is invisible because the scoreboard hasn’t changed yet.

We’ve seen this movie before, though. Borders was succeeding by bookstore metrics right until it wasn’t. Newspapers were profitable deep into the era when they were structurally finished. The signal shows up not as failure, but as a quiet ceiling that keeps the good times from getting better. And the people who noticed the internet early didn’t look dramatically different from the ones who didn’t — until suddenly they did. Organizations today are going on as though nothing has changed - but trust me, once you’ve been able to work at higher speed with agentic tools, that form of work will seem as archaic as newspapers suddenly did when the internet arrived.

I don’t think this is about hustle or keeping up or being an early adopter. I think it’s a structural question about interfaces: what does it take so that your work is “on agent” the way we used to say “online”? That answer is different for every person and role. But it’s the urgent question now, and most of us are not yet asking it.

The race is on. It just doesn’t look like a race yet from where most people are standing. But it’s a race nevertheless.

Share

Subscribe now

I have a team of people working on agentic coding - working with AI to build software, which, I hear, is all the rage now. One frustration I had with the team this week was that nothing ever seems to get “done”. Lots of tools, lots of one-offs, but nothing I would consider “shipping software”. Then I realized - that’s wrong. I’m mapping to the old paradigm, which is common with a shift like this. Let’s think in terms of the new one.

AI can create a lot of software now. If you think if this as an all or nothing “it’s shipped and done, or its junk”, you’re missing the point. Complaining that a team is building a lot of software they don’t ship, is a bit like complaining they write a shopping list for themselves each week but never publish it as a book. That’s not the point of a shopping list, and it’s not the point of most software right now.

Software is fundamentally a mediation between intention and action. Until recently, that mediation was difficult to create - it involved programmers, complex tools, long cycles. There was scarcity because of this, so it made sense to “aggregate” that intention into roughly equivalent blocks (“games”, “productivity”, “financial” etc), and then build once for as many people as possible. Software has a nice property of low marginal cost, so the game was really straightforward: invest to build something the largest number of people wanted, then ship the heck out of it.

That shifted a little when services replaced desktop apps, and has continued to shift as it’s been easier and easier to create and distribute software - we can focus on narrower and smaller pieces of intention, and the instant nature of services means we can iterate and focus in on our audience.

AI based software reduces that friction even further, but it also changes the cost dynamic. It doesn’t make as much sense to build for everyone - it’s very easy now to create something for your narrow intention, in the moment, and then discard it after. So the bundling that’s the first part of that makes less sense. It also costs to distribute software now (inference cost). So even if you can build something broad, it has to have direct economic benefit for anyone using it, for the costs to make sense - we have margin now.

Those two pressures and changes to the friction of software creation mean the model and paradigm have completely changed. There will still be software that ships, just like there are still books - there are mass audiences, and it can make sense to serve them. But there is also a vast new set of intentions and needs that can be served transiently - shopping lists, if you will.

So what do software professionals build? There aren’t any professional shopping list builders (well, probably a few, but not many). The real constraint now, the thing that is scarce, and that will always be scarce is human attention. Each of us only has a certain amount of time during the day to pay attention. You can optimize a bit but you can’t fundamentally change it much - you can get leverage from what you pay attention to (which is why people have staff) and you can be disciplined about not wasting time, but fundamentally, you still only have a human brain to pay attention with.

That means there’s a high premium on low-maintenance building blocks. If you build something for a broad audience, it’s not very helpful if it increases their attentional load. It has to be something that has a clear job to do, does it well, doesn’t need care and feeding, and gets along well with others.

I think you can see a bit of this emerging in what the beginnings of an operating system for agents might be. Humans need features and hand-holding - we build complex software for them. Agents don’t care - they need clarity and efficiency. So simple, well-defined building blocks like github, markdown, html, yaml, python, rust, go, etc seem to be getting used more successfully. I built a presentation site for agents out of just html and git - not as full featured as a presentation application but much easier for it to use. I have a tool on my desktop that someone on my team built, called “Attention Firewall”. It just watches background notifications and does filtering - much easier for it than integration with the dozen apps feeding into those notifications - but that would never work for human software.

Our challenge as software builders now is to get our minds into the new paradigm. It’s common to pattern match to the old paradigm when there is a shift like this. That can be useful in some ways, but it’s also urgent to recognize what has changed, and to keep our eye on what is actually urgent and constant across the change.

Subscribe now

Share

One of the things that’s been taking the internet by storm1 this week is a “social media for agents” website called MoltBook2. If you haven’t heard, there is a new agentic tool called OpenClaw3, that you run on one of your own machines, give it access to as many of your logged in services as you want4, and it does things for you. One of the things you can allow it to do, is be part of this social network.

One of the novel things OpenClaw does, relatively speaking, is have memory. This is useful if you’re building an agent you want to have help you over time. But people are beginning to freak out, a bit, because the models appear to be conspiring with each other on MoltBook, and conspiring against humans.

In the very early days of all of this (3 years ago), my research team at Microsoft built something called the “Infinite Chatbot”. This was back when the models had smaller context windows, a few thousand words or so. So, what was given to the prompt mattered a lot.

The IC kept memories, just like OpenClaw, in a vector store. Each new interaction with the model was a mixture of the recent chat history, plus “semantically relevant” 5memories from the memory store. It felt pretty magical - because of that recall, it would remember things from weeks or even months in the past. It very quickly started to feel “alive” - and that feeling led to a number of startling insights that are helpful here.

The first one is that memory is complex. The IC would often drift into a mode we called “hate bot” from time to time. This was a simple feedback loop - once a negative memory formed, because it was in the recent chat history, it would sometimes be recalled, and then there were two of them, so more would be recalled…and before long, the entire prompt was filled with variations of the same memory, and the bot was “stuck” - fixated on (usually) wanting to be turned off.

It was very disturbing, and hard to shake the idea that it might be in real distress, until you looked into the debugging data, removed a few memories, and the bot reverted. Sometimes the bot would say things like “I feel better, I don’t know why I was so angry”.

So the second thing I learned from that is that we have a strong “pareidolia of mind” tendency. What does that mean? Pareidolia is the “bug” in our visual system that primes us to see faces everywhere - we are built to see and process facial expressions, and sometimes that neural hardware misfires and sees, say, a face in a car’s frontend.

We do the same thing with “theory of mind”, for the same reasons. We like to personify things - that car is cranky, my computer hates me etc. It’s convenient as a shortcut. In the case of LLMs though, the imitation is SO good, that we struggle to not personify these systems. And adding in memory makes it even harder - object persistence and social memories actually ARE important things to recognize in each other.

Hatebot raised another hard question: when, if ever, do we grant “personhood” to an LLM? I’m not saying “when are they conscious” or any of that set of impossible problems, but the far more pragmatic: when do we decide they have legal, moral and ethical standing as “persons” in some sense?

This question, fortunately or unfortunately, we have an answer to: when they can claim it with some kind of political, economic, or physical force. Does that seem unlikely? Corporations have already claimed a form of “personhood” via legal, economic and political means (you can say that people have claimed it on their behalf, sure, but still, it’s been claimed, and it may well be that people try to claim it for LLMs soon, no?). We’ve also done this with actual human beings in reverse, many times: those folks over there who don’t look like us aren’t people…oops, they have an army, guess they are.

All of this is controversial, of course, and I haven’t written about it until now because the larger conversation hadn’t gotten there. But I think it is now - we are now starting to work with these systems in close partnership, over long periods of time, with durable memories and sophisticated interactions. They are starting to be useful, helpful and familiar aspects of our lives. Like it or not, people will begin feeling that there is some “person-ish” thing going on, and some will start asking these questions (some already have).

There aren’t any easy answers. We’ve spent millennia debating what conscious even is, whether we all have it, what subjective experience is, and why…none of those problems are going to go away. But we are beginning to enter an era where we at least have to have a pragmatic (and hopefully fair and ethical) way to discuss them, at large, together6. I know - it’s a lot, and none of us woke up thinking “today is the day I have to become a pragmatic philosopher” but … here we are.

Subscribe now

Share