One of the questions posed by tech industry insiders for the past few years is whether AI will have a “fast” or a “slow” takeoff. A fast takeoff means that AI itself becomes smart enough to accelerate its own development, and that feedback loop goes very quickly. It’s not an unreasonable concern - we’d like to keep control of our technologies, and we don’t always fully manage that even with non-self-improving ones!
It’s a tough thing to debate, since we can’t prove the negative (it won’t happen), and if the positive does happen, it won’t matter (because it’ll take off). But lately, it occurs to me that there might be a few things we could watch for that would be early indicators. And unless I’m wrong, I don’t think we are seeing either of them yet.
The first would be what I call “fast takeoff teams”. This is a team that is using AI based tooling in a compounding manner, or recursively. There are plenty of teams who are getting good acceleration in a linear way, where the “constant factor” of improvement is very high - they can do something in 10% of the time, say.
But geometric or exponential acceleration is different. It would look like “we built this tool that lets us get 2x as much done. Then we used the tool to build another tool that would have been hard to build, and that lets us get 4x done. That tool is so capable that it let us build this really hard tool we thought was impossible, and that lets us get 8x done. Now that tool is working on another one that we barely understand that we think will let us get 16x done…” That’s compounding, using the tools to build an ever-taller stack of ever faster tools.
It could be we are seeing this, and the exponent is more like 1.1 or even 1.05 instead of 2, so it’s not showing up quickly, yet. But I don’t hear much like this from teams - possibly some of the code generation teams are getting there.
The other indication we would see of a fast takeoff would be the training curve bending. The curves are very stable, and logarithmic - each step is 2x more expensive1, but intelligence (for some definition, see below), is increasing. Unless there is some unexpected breakthrough (possible, but again, impossible to prove or predict by definition), we should see that relationship begin to bend if the models are making significant progress towards making themselves smarter, or cheaper at the same level. As far as I know, we aren’t seeing that yet either (I could be wrong with both of these - I don’t exhaustively research the field).
There are lots of reasons this might or might not be true. Some of them are practical - models need to live in the real world enough to have real impact, and maybe we just haven’t connected all the plumbing yet, for example.
One thing that intrigues me though, is that we use words like “intelligence”, and “reason” as though they were well defined. Are they? Certainly, there are things we can build that are really “intelligent”, more than we are, in some domains - the trivial example is always the calculator, but almost any definition of intelligence has trouble excluding most software. That’s why you have to modify it with something like “General” when we talk about AGI.
The models we build are some kind of intelligence, just like all software is, at least in the sense of “can do useful things with information”. These are broader in some ways, and sometimes that breadth fools people into thinking they’re broader still. This is just an occupational hazard, just like we get emotionally involved with fictional characters on the screen and in books, we are experiencing the same confusion with these very powerful new tools.
It’s easy to look at what they can do, assume they can do all the rest of the things we can, and expect that they will be us, but faster and electric powered. And maybe they will, probably they will someday (the human brain uses roughly 20W of power, so there is clearly a long way to go, to improve on current LLMs). But right now, we don’t seem to be seeing the compounding we’d expect from a fast takeoff, and I suspect that’s because there are still some missing pieces (like iteration, experimentation, and more robust, continuously integrated and nonlinear memory).
If you want to understand this more, and see some graphs, Ethan Mollick has a good article on it: Scaling: The State of Play in AI - by Ethan Mollick
Thanks Sam, good thoughts. Regarding exponential teams -
a) AI and AI harnesses (e.g. coding agents) are not yet uniformly advanced for all stages of product development. Coding agents are only now becoming truly useful (probably last six months). Other agents (e.g. for Product requirements) are still in development.
b) Exponentials often look linear at first.
c) As a fact people are building AI-related tools with AI. Case in point - I built a Roo mode-builder with Roo.
d) Most organizations are not building AI, so can't be exponentially. There's evidence of continuously increasing speed at some of the frontier labs that are building the core models.
I think looking at the individual brain is the wrong benchmark. Humans succeed at the level of culture and civilization, these bootstrap our development from conception. I think AI gets subsumed into the infrastructure of our civilization as augments but don't achieve a stand-alone viability.