I suspect this newsletter is going to touch on AI for a while. I typically write things that are in reaction to what I see during the week, and that’s mostly what I’m spending time on these days - I’m back to building products with a small team rather than spending more of my time coaching (and it’s wonderful!).
There are lots of debates about whether the current LLM approach will lead to “true AGI” (though I suspect we are going to see a lot of “no true Scotsman” thinking about what intelligence is, as models get better). But I do think there is an important shift happening in the nature of coding itself because of these models: the shift from syntax to semantics.
One of the arguments about LLMs is that they are just “stochastic parrots” that have no real understanding of the world, because all they do is understand word probabilities. I think this is a shallow argument. Natural language embeds a great deal of understanding about the world, almost by definition. If I say “The monkey ate the”, you are likely to complete that with “banana” or “grape” not because you have done some math on a fundamental property of the letters in the sentence (aha! the binary is 1 mod 13 so the next word must be 0 mod 11! or whatever), but because there is some knowledge of the world and the culture you live in, embedded in our usage of language.
We can begin to see this in some of the more advanced models. There are many examples of errors and misunderstandings, but it’s also the case that we can see places where chatGPT and others have some grasp of meaning - in other words semantics.
For the entirety of the computer industry, we have been forced to deal with process and syntax. We have to mediate between the complexity of the real world and the rigidity of the digital - we build schemas and algorithms. But in our daily lives, we don’t live that way - we are iterative, goal-driven and reactive. We deal with semantics instead of syntax, goals and intent instead of rigid process, and we do iteration and react instead of having fixed algorithms.
We aren’t all the way there yet, by any means, but this is a fundamental shift in how software will be conceived of and built, for the rest of our lives. This is the “second version” of the computer industry. Of course, there have been folks working in this direction with some results for years, but now we are at the point where this fundamental shift is becoming widely accessible.
This is very analogous to the early internet, which existed for many years in smaller form until enough new technologies (better packet routing, faster computers, HTML and the browser) came together to make it widely accessible. There was a quick period of excitement, some early missteps, and then a long period of really figuring out how to make everything work.
In the early days of the internet, there was a debate about whether things like eCommerce would work, whether it was worth it for a business to have a web site, etc. Now, it’s fairly obvious that if you aren’t accessible to the internet, at some level, you don’t exist. The same is going to happen for software and companies in the age of semantics and AI - anything that isn’t available to AI, that can’t be managed easily through non-brittle goals and iteration, will start to seem “invisible”.
So, don’t be fooled by early mistakes, reporters or others who make a living writing scare stories, or all the other flavors of “why not?” that’s out there. This is a fundamental shift that we will be working through for quite some time, and it will reorder virtually everything about the technical industry at some level, eventually.
I remember some years back that programmers would consider the syntax and the semantics the same, and consider this discussion purely academic. Now NLP/LLMs mean nontrivial work can be done without formal syntax and suddenly the debate has flipped. We’ve always had errors and inflexibility in the syntax is semantics approach but they call them bugs and i implemented features. The possibility now is flipping that and being able to describe semantics without syntax errors, change or extend semantics without a 3 year, 1 year, or even 1 week release cycle. There are many other benefits too.