There is a debate going on right now about whether AI is “worth it” for various tasks. I wrote about this last week, and it’s been on my mind again this week. This time, let’s look at some anecdotal signals instead of trying to derive from first principles.
When Amazon first appeared, there was a lot of debate about the value of ecommerce. No one was really used to it, shipping and returns were hard, logistics was unsolved, and so on. I remember reading lots of very serious articles on how the economics would never work, and indeed, the companies that implemented it poorly (hello Pets.com) failed.
But Amazon started out with what seemed like a really smart premise: we’ll give you access to every book published. In the modern age of streaming, kindles and all kinds of online content, this doesn’t seem that useful, but in that era, when bookstores were a disorganized mess, hard to search, inconvenient to get to, and slow to order from, it was a real improvement1. And whether or not the economists and pundits recognized that value, end users did, and it powered that business.
You could think of LLMs as “Amazon, but for thought”. Suddenly you have access to, more or less, all of the thinking, in one convenient place. Experts and pundits can talk about whether that’s really more efficient, but users are speaking, and I think there is real value there, whether it’s “officially” captured or not.
Here are my anecdotes: the lead engineer on one of my teams is currently spending about $200 a day in inference costs to do his job, and they’re (to my eye) much more productive than before. We don’t regret the spend and are looking for ways to usefully increase it.
A good friend is a professor at a local university estimates that they spend roughly $500 a month on one of the AI services to help with their research - and they are doing things they couldn’t or wouldn’t have done before. I don’t know of any other service where anyone I know spends like that!
And the pattern of “I get to do a new thing” is showing up in other places. I have “non-technical” people producing ideas and prototypes they wouldn’t have before. It’s hard to measure the productivity of that, because you’re dividing by 0.
I don’t really have much direct insight but I don’t know of any big company or datacenter that has "too many” GPUS right now, or even any that aren’t utilized. As far as I can tell, everyone is using it as fast as they can build it.
And there definitely seems to be some kind of “Jevon’s paradox” going on - token cost came down 100,000X over the past few years but usage has gone up even faster. That’s bad if you were depending on the reduction in cost to fix a margin problem, but really good if you are trying to understand market demand. It’s not surprising - as the models get smarter, we use them. Who wants the dumber assistant?2
The last two times this happened were the internet buildout, and the mobile one. I remember the exact same feeling - we almost couldn’t build capacity fast enough to satisfy demand. If anything, AI adoption is more rapid.
Value isn’t always crystal clear. It’s easy to measure a decrease in cost or time - but that kind of “value add” isn’t really interesting in the long run, because you can’t get below 0. What is interesting is new value. But what is that? We watch a lot of content that would have been thought to have no real value 20 years ago and wouldn’t have been produced. Does that “not count” somehow?
I don’t think so. Value (once you get past food, shelter and safety) is doing the things we want to do. The “Amazon of thought” that is being built is allowing a whole lot of new things to happen, in a lot of new ways. I suspect that creative energy and demand is going to continue for a long, long time.3
Don’t come at me - I actually love bookstores and spend a lot of time in them and support them. But you have to admit, while they are wonderful, and good for discovery, they aren’t particularly efficient.
Though there are almost certainly opportunities to arbitrage cheaper/smaller agents that are going unused right now.
I don’t usually disclaim this but: this letter (and all of them) is really just my personal opinion, and not any kind of official Microsoft statement or position. This is just me, as a technologist and technology follower for 40+ years, remarking on what I seem to be seeing. Take it with a grain of salt.