Does Agentic RAG Actually Retrieve Better Evidence? I Tested It on 298 Questions | Towards Data Science
The author tested whether agentic retrieval-augmented generation, where an LLM agent rewrites queries and decides what to fetch, actually finds better evidence than plain search and reranking. The whole setup is reproducible from a GitHub repo, with 298 reviewed questions and saved results.
- The test pits agentic RAG against plain search and a reranker on 298 questions, with Claude models writing the questions and running the agent.
- Results are reproducible: the code, questions, and saved outputs for every run are published, so you can check the numbers yourself.
- Reciprocal-rank fusion is the core merge step, combining several ranked lists of retrieved documents into one.
The excerpt we have is thin on results, so I can only summarize what it sets up. The author ran a test of agentic retrieval-augmented generation against 298 questions and published the code, the reviewed questions, and saved results in a GitHub repo (agentic-rag-evidence-test). The question set was written by Claude Sonnet 5.5, and the agent pipeline runs on Claude Haiku 5.5, which handles both query rewriting and the agent loop. Plain search and the reranker run locally with no API key, so a reader can reproduce the non-agentic baselines on a laptop and only pay for the Anthropic calls if they want the agentic side.
The core mechanism shown is reciprocal-rank fusion (RRF), used as the merge step for combining several ranked result lists. Each document gets a score of 1/(k + rank) from every list it appears in, and the scores are summed, so a document that ranks reasonably well across several query variants beats one that ranks first in only one. The `k` constant defaults to `config.RRF_K`, which is the usual smoothing parameter that stops top ranks from dominating. That fits the agentic setup: the rewriter produces multiple queries, each gets retrieved, and RRF fuses the results before anything reaches the answer step. The question the article is really testing is whether that extra retrieval work, with the rewriting and fusion, buys better evidence than a single well-tuned search pass.
The excerpt does not include the headline numbers, so I can't tell you whether agentic retrieval wins, loses, or wins only on certain question types. For that, you'd need the sections after the setup, which presumably report recall or evidence-hit rates against the reviewed answers. The author's track record here is worth noting: they write about reproducing retrieval methods end to end, failures included, not just the happy path. That makes the saved run logs and the 298-question set the part most worth reading closely, since they let you check the claim rather than trust the write-up.