The Orchestrator's Tax

Open original

Martin Fowler shows that in long-running multi-agent workflows the real cost isn’t parallel execution but context pollution in the orchestrator’s working memory. He recounts how status polling and redundant file orientation dumped massive transcripts into the main thread, competing for attention. He then proposes simple rules—like merging overlapping tasks and avoiding unnecessary polls—to keep only essential information in context.

  • Status-check prompts that dumped full JSONL transcripts into the main thread, not parallel execution itself, were the real token/attention sink in a multi-agent Claude Code session.
  • The actual bottleneck in long multi-agent runs is the orchestrator's working memory getting polluted, since tokens are a one-time cost but context pollution degrades focus for the rest of the session.
  • Running adjacent subagents independently meant each paid the full cost of learning the same architecture from scratch, wasting effort that duplicate orientation could have avoided by grouping tasks with shared context.
  • The fix is partitioning work by "cognitive locality" (grouping tasks that share a mental model) and having subagents return only refined conclusions rather than raw reasoning, keeping noise out of the orchestrator's context.

I hit a wall in a Claude Code session on a .NET refactor when four parallel subagents started spitting out results out of order. At first, I assumed the trade-off was obvious: pay a bit more in token usage to cut wall-clock time from twenty-five minutes down to around twelve. But when I poked deeper—asking the orchestrator to critique its own decision—I found something else eating my budget: those status-check prompts that dumped entire JSONL transcripts back into the main thread. Tens of thousands of tokens of intermediate reasoning kept piling into the orchestrator’s context, and every subsequent prompt had to sift through that noise. Breaking the session’s costs into pieces changed the story. Two subagents were working on adjacent parts of the same response pipeline, each paying the full “learn the architecture” price independently. Another agent used git stash and git stash pop while its siblings wrote elsewhere in the repo, introducing risks you don’t face in a single-threaded session. I tried ranking these overruns—orientation duplication, unsafe repo ops, noisy polling—but none of those measures got to the heart of the problem. What really matters in a long-running multi-agent workflow is the orchestrator’s working memory, not raw parallel speed. Tokens are a one-time fee. Context pollution lasts the whole session and makes it harder for the model to focus on the task at hand. Bigger context windows won’t help if most of that space is filled with irrelevant dumps. The point of subagents should be to keep noisy, disposable reasoning out of the main thread, returning only the refined conclusions the orchestrator needs. That means partitioning work by cognitive locality—grouping tasks that share the same mental model—so you avoid rebuilding understanding multiple times.

Ask about this article