@0xMortyx - RAG vs Kimi K3
@0xMortyx on X A data scientist ran a blind test pitting RAG against Kimi K3's full 1M-token context window on the same 12 questions. Long context scored perfectly on answer quality but cost about 16 times as much and ran roughly 3 times slower, so the post argues the right choice depends on corpus size and query volume.
- Long context answered all 12 questions perfectly, while RAG was near-perfect but weaker on completeness.
- Long context cost about 16 times as much and took roughly 3 times as long per question; at 12,000 queries the cost gap reaches about $3,800 versus $230.
- Skipping RAG is only backed by evidence when the corpus sits well under a fifth of the window; the test never covered corpora filling 60-80% of it.
- For tasks spanning many unconnected sources, the author points to Kimi K3's Agent Swarm and a connected context graph instead of either approach.
A blind test pitted RAG against Kimi K3's full 1M-token context window on the same 12 questions, same model, graded without knowing which path produced each answer. Long context won on quality, scoring 12 out of 12 perfectly, while RAG was near-perfect but less complete. The catch: long context cost about 16 times as much and ran roughly 3 times slower per question. At 12 questions the gap is a few dollars. Scale the same corpus to 12,000 queries and the difference is roughly $3,800 versus $230, so query volume, not corpus size, is what really drives the cost decision.
The author's central argument is that RAG, long context, and a connected "context graph" each suit a different shape of problem. RAG was a workaround for models that could only see a few thousand tokens, and that constraint mostly disappears when your material fits comfortably in the window. The test corpus was 127,068 tokens across 32 articles, about 12% of the window, and at that size the model answered everything perfectly. The article is clear that this only tells you about corpora below roughly a fifth of the window; the test never covered 60 to 80% fill, where selection becomes necessary again. The author also flags three practical traps: K3 is a reasoning model whose thinking tokens count against the same output limit as the answer, so a low limit can produce a silent empty response; prefix caching, which should cut repeat-call costs, only triggered on about a third of calls with no obvious pattern; and on the hardest questions, RAG's retrieved chunks sometimes couldn't answer at all, so an explicit instruction to admit gaps was essential to get an honest "I can't answer that" instead of a confident guess.
The last section moves past single-corpus querying to tasks with no defined knowledge base, like competitor research or market scans drawing on hundreds of unconnected sources. Here the author points to Kimi K3's Agent Swarm, where an orchestrator spawns sub-agents without predefined roles, running up to roughly 300 of them and about 4,000 coordinated steps, then stitches their outputs into one connected structure rather than a pile of separate answers. The practical advice is to choose the tool per task instead of per project: RAG for frequent, narrow lookups on a large stable corpus, long context for occasional deep reads of a corpus that fits the window, and agent swarms with a context graph for wide research where connections between sources matter. The article's own caveat is that these benchmarks come from one author's single experiment, so the sensible move is to run the same blind comparison on your own data before committing.