Agent memory as a file format

Open original

Memoryfield proposes treating agent memory as a simple data format (Markdown files plus optional vector index) rather than a complex pipeline system. It avoids the lock-in, overhead, and retrieval problems of existing memory systems by letting AI agents write prose directly and search semantically instead of traversing knowledge graphs.

  • Existing agent memory systems fail because they're vendor lock-in platforms, unnecessarily complex (requiring multiple databases), or strip context from facts—memory should be data, not process
  • Semantic search with parallel reads beats graph traversal: jumping directly to relevant pages requires 2 tool calls instead of N+1, finds information more reliably, and reduces noise in context windows
  • The format scales with model improvements because agents can natively work with Markdown, SQLite, and bash rather than navigating fixed APIs, so better models automatically discover more creative uses
  • Memoryfield is transport-agnostic (zip, local files, S3, GitHub, HTTP) and spec-based to prevent vendor lock-in and keep memories portable as agents and models change

Most AI agent memory systems are broken in predictable ways. Some lock you into proprietary platforms designed by labs trying to escape the competitive API business. Others are absurdly complex—one needs pgvector, Neo4j, and a dedicated LLM just to filter what's worth remembering. A third type strips information into isolated "facts" and graphs, leaving agents confused about context. The core problem: they all treat memory as a process requiring pipelines and stages, when it should be treated as data. Cal Paterson proposes memoryfields—a file format that's essentially a ZIP containing Markdown pages with optional YAML metadata and a SQLite vector index. Agents write memories directly as prose (around 2000 tokens per page) rather than having systems extract, chunk, and reprocess human documents. The format solves three specific problems. First, prose beats fragmented facts because agents already write naturally in Markdown—no need for mechanical processing. Second, semantic search beats knowledge graph traversal. Walking a graph forces serial tool calls (2-3 seconds each) through irrelevant pages, and agents miss information that isn't perfectly titled. Memoryfields use vector search to jump straight to relevant pages in parallel, requiring at most two tool calls total. Third, the low-mechanism design (just a file format) lets agents invent their own access patterns rather than navigating API mazes. As models improve, they discover new ways to use the system—like running bash commands or querying inline CSV files—without requiring format changes. The practical constraint of 8KB per page (roughly 1,300 words) actually works in favor of clarity. If you need more detail, you write more pages. Agents don't struggle with this. The whole approach inverts the usual complexity: instead of building elaborate systems to make legacy documents work with AI, memoryfields let agents write their own memories in a format they're already comfortable with, then retrieve them via straightforward semantic search. It's deliberately simple enough to outlast the current model frontier.

Ask about this article