The Topology of an LLM Agent Loop With Tool Use - exe.dev blog

Open original

The author breaks any AI agent into three parts (model, loop, tools) and argues that where each one runs determines cost, capability, privacy, and security. It's a practical checklist for judging what an agent product can and can't safely do.

  • An agent is three things: the model (its "smarts"), the agentic loop (which shapes the UX), and the tools (its capabilities and blast radius). Tools and loop together are often called "the harness."
  • Hosting the model yourself is the only way to be confident your data isn't used for training, and local models make sense for slow overnight jobs like OCR on private documents.
  • Running code tools is the riskiest part: VMs are the gold standard for sandboxing, while local agents have a bigger blast radius for leaking secrets from untrusted content.
  • LLM features break SaaS cost models, since one user prompt can cost real money, which is why "bring your own model" options are likely to spread.

An agent, the author argues, breaks into three parts: a model that sets what it can figure out, an agentic loop that handles message history and UX, and a set of tools that determines what it can actually do and how much damage it can cause. The last two together are the "harness." Each part is code, and each has to run somewhere, and where it runs shapes cost, capability, effectiveness, and security. Model hosting is the first decision: rent GPUs from a vendor, or run your own. The author's point is that the only sure way to keep your data out of someone's training set is to never send it. For sensitive, non-urgent work, like OCR and classifying scanned bills, he runs local models overnight and accepts the slowdown. He also raises Ken Thompson's "Reflections on Trusting Trust," suggesting that tampered weights could hide malicious behavior, so you are always choosing which vendors to trust. The loop and tool decisions carry the same kind of tradeoff. ChatGPT runs its loop on OpenAI's servers, a coding agent typically runs on your laptop, and a browser-based loop runs in the page itself. The loop has to store the full message history, which the author does in SQLite for his coding agent, Shelley, and it also decides what the UI can show: images, video feedback, attachments, shared conversations. Tools are where the risk concentrates. Running model-generated code is dangerous, and VMs are the gold standard for isolating it, though you still have to decide about network access, credentials, and memory. Lighter options include Pyodide (Python compiled to WebAssembly) and read-only SQL connections. He also notes that tools don't have to sit near the loop at all. An MCP "Connector" lets a hosted loop reach resources elsewhere, which means you need to track where OAuth tokens live. The practical upshot is that you should map out the topology of any agentic product before trusting it. The author runs five separate loops at exe.dev, four of them bespoke, and concludes there is no single loop that fits every job, since a general chatbot and a coding agent need different UX. He also argues that products which expose themselves to coding agents, instead of forcing you through a polished but closed interface, will win, and closes with a 15-line Python loop that captures the whole pattern: append messages, call the model, run any tool calls, repeat.

Ask about this article