More on the topic…
Most AI agent memory systems are broken in predictable ways. Some lock you into proprietary platforms designed by labs trying to escape the competitive API business. Others are absurdly complex—one needs pgvector, Neo4j, and a dedicated LLM just to filter what's worth remembering. A third type strips information into isolated "facts" and graphs, leaving agents confused about context. The core problem: they all treat memory as a process requiring pipelines and stages, when it should be treated as data. Cal Paterson proposes memoryfields—a file format that's essentially a ZIP containing Markdown pages with optional YAML metadata and a SQLite vector index. Agents write memories directly as prose (around 2000 tokens per page) rather than having systems extract, chunk, and reprocess human documents.
The format solves three specific problems. First, prose beats fragmented facts because agents already write naturally in Markdown—no need for mechanical processing. Second, semantic search beats knowledge graph traversal. Walking a graph forces serial tool calls (2-3 seconds each) through irrelevant pages, and agents miss information that isn't perfectly titled. Memoryfields use vector search to jump straight to relevant pages in parallel, requiring at most two tool calls total. Third, the low-mechanism design (just a file format) lets agents invent their own access patterns rather than navigating API mazes. As models improve, they discover new ways to use the system—like running bash commands or querying inline CSV files—without requiring format changes.
The practical constraint of 8KB per page (roughly 1,300 words) actually works in favor of clarity. If you need more detail, you write more pages. Agents don't struggle with this. The whole approach inverts the usual complexity: instead of building elaborate systems to make legacy documents work with AI, memoryfields let agents write their own memories in a format they're already comfortable with, then retrieve them via straightforward semantic search. It's deliberately simple enough to outlast the current model frontier.
Questions about this article
No questions yet.