LLM Renderman
(Credit to John Maeda for this thought)
When I was starting out in the computer industry, I was very interested in (though not active in) computer graphics. If you’ve grown up in the world of photo realistic video games and seamless CGI effects, it’s hard to imagine but there was a long period of time when the potential of using digital effects was clear, but the reality was still missing. I remember eagerly watching all of the SIGGRAPH films each year to see how far the state of the art had advanced.
At first, rendering was so expensive that only images and then short videos were possible. Pixar started to blow our minds with videos that, while not realistic, were believable enough to be watchable and enjoyable. I remember being amazed by Luxo Jr - how they had managed to breathe life into a simple object. Watch it, if you haven’t seen it, and try to think that that was amazing state of the art at the time.
We all wanted to see a fully digital movie, but for a long time it was too expensive and too hard. Pixar finally made Toy Story, which works really well as a movie, but it’s a total cheat: they still couldn’t do things like skin and hair realistically enough, so they wrote a story about plastic toys, with a setting that supported a cartoonish, unrealistic look - and got away with it fantastically.
I think there are parallels here to what’s going on in AI right now - at least with LLMs. Think about the themes above: the potential is there but not realized yet (we have very grand dreams of agents, but very little to show for it yet), it’s too expensive and slow technically still (though improving!), there is lots of technical advancement because of the excitement, and we have interesting things that are not quite realistic - we have to choose our domains carefully.
We’ve been saying that AI feels a lot like the dotcom boom, and to some degree it does. But it also feels like those early days of the CGI industry in many ways. And I suspect it will evolve in exactly the same way: we will get better, cheaper, faster tech, we will gradually build more and more ambitious and complicated solutions that will seem more and more realistic, and we eventually will become so adept at it that AI will become seamless and unremarkable (when was the last time you noted how good the CGI is in a movie? It’s just expected now. Can you even point at a particularly interesting effect in the last 5 years? Not really - everything is possible, and perfect).
The tools we use now are like early graphics tools - Renderman and Video Toaster for example. We should expect a lot of evolution and development. It might take 20 years, like the CGI industry did, or it might be much faster (I suspect faster - we’re better at it now). But just because you see an overly ambitious “bad render” that doesn’t work out (I’m looking at you, Polar Express) doesn’t mean the whole thing isn’t progressing. In fact, it’s likely that the next team to try the same thing will get it to work.