@Oliviacoder1 - MIT Recursive Language Models

Open original

@Oliviacoder1 on X Three MIT CSAIL researchers (Zhang, Kraska, Khattab) propose Recursive Language Models, which store a long document as a variable outside the model's context and have it search and read only the relevant parts, spawning sub-models to handle sections in parallel. The post claims this handles inputs about 100 times larger than a model's native window with better accuracy at similar cost, and the code is open source on GitHub.

  • Standard frontier models score near zero on the hardest long-context reasoning benchmarks, and GPT-5 couldn't solve even 10% of problems requiring tracking code history beyond 75,000 tokens.
  • RLMs keep the full document intact and queryable instead of summarizing or chunking it, and reportedly handle inputs up to 10 million tokens with double-digit gains over alternatives.
  • The cost per query is reported as comparable to or cheaper than large-context calls, and the code is free on GitHub as a drop-in replacement for existing LLM API calls.

MIT CSAIL researchers Alex Zhang, Tim Kraska, and Omar Khattab posted a paper on arXiv on December 31, 2025 (arXiv:2512.24601) proposing Recursive Language Models, or RLMs. The core problem they target is "context rot": as you stuff more text into a model's context window, its accuracy on material already inside that window degrades. Facts blur and information in the middle gets lost. The industry's standard workaround, retrieval-augmented generation (RAG), chunks documents into a database and fetches pieces on demand, but the retriever has to guess what matters before the model has read anything, and chunking breaks the relationships between distant passages. The article's framing is that the whole context-window arms race, which went from GPT-3's 4,000 tokens to Gemini 2's 2 million, rested on the assumption that more context means better performance. The RLM approach is to stop loading the document into the model's context at all. The full text is stored as a variable in a Python environment, and the model is told the variable exists and how large it is. The model then writes code to search, slice, and navigate the text with things like regular expressions, pulling only the relevant sections into its active window. It can also spawn smaller sub-model calls to read and analyze those sections in parallel, with a root model synthesizing the results. Nothing is summarized or discarded, so the original text stays intact and queryable. The analogy the author uses is a human expert navigating a large archive rather than memorizing it. The reported results are the part worth checking against the paper itself. The article says frontier models such as GPT-5 scored near zero on hard long-context benchmarks, with GPT-5 solving under 10% of problems requiring tracking code history beyond 75,000 tokens, while RLMs achieved double-digit gains over alternatives and handled inputs up to 10 million tokens at comparable or lower cost per query. The code is open source on GitHub (github.com/alexzhang13/rlm), so the claims can be tested directly. Keep in mind that the piece is a promotional-leaning X post, and its "walking away from the battlefield" framing and claims that the context wars are "over" go well beyond what a single benchmark paper can establish. The benchmarks are specific long-context reasoning tasks, and whether RLMs generalize across real workloads is an open question. The article also mentions Prime Intellect's plans to train models to manage their own context with reinforcement learning, which is described as future work rather than a result.

Ask about this article