"bots" are docs, again

There is a strong tendency to get confused by the behavior of LLMs. They seem human in some ways, they seem to be “present” - so we tend to anthroporphize them. We call them “agents” and subconsciously expect them to behave in ways that humans do. For example, we expect them to know how to keep a secret - if they know something about us that’s not appropriate in some context, we don’t expect them to spill the beans the way we would expect human discretion.

The problem is, though, that LLMs are not robustly secure on their own. There aren’t really any known techniques to defeat jailbreaking. If the LLM knows something, it can be extracted by skilled practitioners. This isn’t necessarily the indictment of the tech that some think it is - leave an open SQL database on the web and see how that goes. Nothing is really secure by default, we have to build for it. But with LLMs, that tendency to personalize them subconsciously is dangerous - we want to use them in ways that aren’t robust and safe.

A good example of this is agents - the “bots” in the above. It’s possible to read a lot of fantasy fiction right now in the business press about things like agents that know everything about you and that go out on the web on your behalf, holding your credit card or other credentials, with a hopeful naive look on their virtual face. Again, put that stuff in a naked database on the web and see how it goes. This is the simple flavor of the problem but there are other variants - what do we do in a shared group project where, presumably, there aren’t any bad actors but there are legitimately different levels of permissions and access? How do agents interact in that environment? How does sharing work, and permission changes, and all of that? I won’t do the whole list here, but you can easily imagine it if you try.

It’s a tough problem, but we’ve already built a solution for it that billions of people are familiar with: document sharing. We have exactly the same set of problems and habits here. Imagine having a single document with your entire work and professional life in it, and telling someone you don’t know very well (or someone you do, even), “just read pages 112-114, don’t look at anything else. It’s pretty absurd! That’s what we’re doing when we say “just talk to my everything, know-all, lifetime companion agent about only this topic”!

What we do in the world of documents is what we should do in the world of agents - we segregate information into containers (files, documents, etc) that each have permissions. “Bots are docs” means that we should do that with agentic memory too - instead of one big pool of memory, we should have separated, permissioned pools, with document/file semantics, for each use of an agent.

Sure, this can make some scenarios harder - sometimes you want global memory! But that’s solvable too - memories can stack in the agent world in a way that documents don’t stack. You look in multiple pools, perhaps using a global one by default the same way we have default templates for documents. More importantly though, making this explicit lets the end user understand it. Understanding means there are fewer places where the user can accidentally expose data they meant to keep private. Again, we have 20+ years of this habit built - we should use it!

We do want LLMs to behave in agentic ways. But it’s important to remember that they are a tool, and a technology. They might imitate human behavior in some ways, but they don’t completely have human context, sensibilities, and judgment. For now at least, if we want robust, secure agents, “Bots are docs” or something like it is an important piece of the puzzle.