There is no answer without a question

Or is there?

Curiosity is a hard thing to cultivate in yourself. It’s easy to settle into the familiar - efficient, even. Habits and patterns are natural things for all of us, and being curious, discontented, and trying new things is often hard.

But curiosity, and its cousin discontent, are central to being an engineer, scientist, or creator. You can’t get an answer to a question you haven’t asked yet, of course. Or can you? It seems pretty basic - the flow of the world is “ask question, get answer”, not “find answer, look for question”.

But it does sometimes work that way, if you’re open enough to it. Sometimes we get anomalies - things that don’t quite make sense or are unexpected. I like the story of the person who invented microwave ovens noticing that the chocolate bar in his pocket melted when he stood near a microwave antenna (also - yikes). Sometimes you find the answer first, and then questions suggest themselves (why did that happen? what else can I do with that?).

I think there’s a bit of a danger with LLMs, in that you rarely, if ever, get an answer without a question, and you have to be very, very careful about how you ask the questions you do ask. Sadly, LLMs are really only replying with the knowledge we’ve given them, and that knowledge is neither complete nor correct in its entirety. So the answer you get from an LLM is always suspect, sometimes more or less depending on the quality of real-world knowledge in that particular domain.

But that’s ok, I think. The world itself is fuzzy and confusing. Physics can’t ever really lie, but it can do very unexpected things. The history of quantum mechanics is a good example of this - weird answers coming out of observations leading us to propose weird questions and weirder theories. The ability to notice that weirdness and methodically interrogate it is critical to being a scientist. The ability to notice and ask hard questions about the irritations of the world is what makes a good engineer, or entrepreneur.

All of us now will have to learn to be skeptical questioners. LLMs are too powerful to ignore - they’re a great tool in many ways and getting better all the time. But like any advanced tool, they are dangerous if not well understood or misused. They’ll also happily waste time on the wrong question if you ask something badly, or ask for something you don’t quite want.

All of this is becoming more acute as agentic systems get more common. Everyone is working on getting more leverage out of our interactions with them - as we should be. Human attention is the scarce resource now, so we want AI to do lots of work per unit of our attention. But that work needs to be effective - it has to produce a good outcome. And that boils down to asking the right question at the start.

There’s no (good) answer without a (good) question.

(I am going on vacation for the next two Sundays. It’s remotely possible I’ll feel like writing something, but I likely won’t. Summer!)