A fun thought on optimization

From a friend of mine (and damn good system thinker) Tim Bozarth, in honor of Dune 2 coming out soon:

"I must not pre-optimize. Pre-optimization is the mind-killer. Pre-optimization is the little-death that brings total obliteration. I will face my pre-optimization. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the pre-optimization has gone there will be nothing. Only the product will remain."

It’s a good line, and a good reminder of an old principle that seems to be coming forward a lot these days: don’t optimize early. In the AI world this manifests a lot as people worrying about cost (or sometimes latency) before they have a working design. This is producing a mindset of scarcity in terms of using inference calls to solve problems. Instead of a good modular design with many small reliable calls, we see a lot of large “prompt engineering” designs that are fragile.

But it’s also easy to get this when you’re designing platforms and products. It’s easy to think “well, I might need this feature, let’s design it in now”, before you really understand what the product will be about. That’s a good way to get distracted at best, or have an unworkable design that has to be redone later, at worst.

It’s tough to balance this because you DO want to think about the rest of the problems in your design space and accommodate them as much as you can as you build. It’s tough to have hard and fast rules, but one good guide is to think about how much effort it will be to change something later instead of pre-optimize. You’re making an “expected value” bet that the effort now is less than the effort later times the probability of actually needing to do the later work (the probability of doing the “now” work is 100% if you do it). That’s not a bad way to think about the tradeoff, even when there’s uncertainty - and you can add a bias factor (say 2x) so that you only wind up dealing with contingencies where the expected payout is large enough that you are likely to really come out ahead.