Factory Research developed an evaluation framework to assess how well different compression strategies maintain context during long-running AI agent sessions. They tested three methods—Factory's anchored iterative summarization, OpenAI's compact endpoint, and Anthropic's Claude SDK—across various scenarios like debugging and code reviews. Their findings show that structured summarization retains more essential information than the alternatives, which can lead to better performance when the agent encounters memory limits. Long sessions can produce millions of tokens, far exceeding what AI models can handle. The usual approach of aggressive compression risks losing key details, which can result in agents forgetting critical information or repeating tasks unnecessarily. Factory's method focuses on preserving information by organizing summaries into specific sections, ensuring nothing vital is omitted. Their evaluation used probe questions to test how well agents remember details, like error messages and file modifications, after compression. The grading system evaluates responses on accuracy, context awareness, and completeness, among other dimensions. These measures are particularly important for coding agents, where forgetting a modified file or misremembering a function can lead to errors. Factory's structured approach outperformed the other methods in preserving the necessary context, making it a more effective choice for maintaining productivity in complex tasks.