A blind test comparing RAG to Kimi K3's 1M-token context window shows long context won on answer quality but costs 16x more and runs 3x slower. The real answer isn't one or the other—it's picking the right tool based on corpus size, query volume, and task shape.

• Long context beats RAG on quality when your corpus fits under 20% of the window (tested with 127K tokens), but the cost advantage flips hard at scale: $3.82 vs $0.23 for 12 queries becomes $3,800 vs $230 for 12,000 queries on the same data.
• Query volume, not corpus size, decides the economics; corpus size decides accuracy. A small, frequently-queried corpus still favors RAG despite worse completeness.
• Three distinct tools exist for three task shapes: RAG for frequent narrow lookups on large stable corpora, long context for occasional deep reads of one corpus, and Agent Swarm/context graphs for wide research across many unconnected sources where connections matter.

Google engineers demonstrate how AI systems are evolving from basic retrieval-augmented generation (RAG) to graph-based architectures that handle more complex reasoning and multimodal tasks. The 90-minute workshop walks through building production agent stacks with semantic graph retrieval and specialized agent orchestration.

• RAG is being replaced by graph-based approaches that organize context semantically rather than just retrieving relevant documents
• The production stack involves extracting graph context and orchestrating multiple specialized agents to handle different tasks
• This represents a concrete shift in how companies are building AI systems—from simple retrieval to structured knowledge representation

PixelRAG converts web pages, PDFs, and images into screenshot tiles, embeds them using a fine-tuned Qwen3-VL model, and builds vector indexes for retrieval-augmented generation. It provides a CLI (pixelshot), pipeline commands for chunking, embedding, indexing, and serving (FAISS or Qdrant), plus a live API and a Claude plugin for visual search.

• PixelRAG ditches HTML parsing entirely, indexing screenshot tiles of pages/PDFs via a fine-tuned Qwen3-VL embedding model to preserve tables, charts, and layout
• Ships a pre-built index of 8.28 million Wikipedia articles, usable via a free live API or a ~217GB local download
• Includes a Claude Code plugin ("pixelbrowse") that lets Claude screenshot live pages and read diagrams/tables visually on request
• Supports both FAISS (simple/local) and Qdrant (disk-backed, quantized, multi-server) as swappable backends for scaling

The article shows how packing specialized, retrieved knowledge into smaller AI agents can match or beat huge frontier models. It explains a structure—raw source extractions, concept entries, theses and a startup primer—plus embedding‐powered retrieval to feed just the right context at query time.

• Structured retrieval (10,000 pages → 381 concept docs + 54 theses via hybrid BM25/semantic search) let smaller models match or beat frontier models on specialized tasks.
• The same pipeline replicated across a dozen domains, from finance to rare medical research to corporate policy.
• A locally-run Qwen model with this "knowledge agent" harness performed comparably to Claude Opus, at zero cloud cost.
• Embedding costs were trivial (under a dollar for thousands of documents using BGE-M3 or OpenAI's text-embedding-3-small) before moving entirely to local hardware.

PixelRAG skips HTML parsing by taking screenshots of pages and indexing image tiles with a vision-language model. It preserves tables, charts, and layout lost by text extractors, outperforming a top text-based RAG by 18.1% on Wikipedia and offering a live-page Claude Code plugin.

• PixelRAG indexes screenshots of pages instead of parsed HTML text, using Qwen3-VL-Embedding over image tiles with a FAISS index across 30M+ Wikipedia screenshots.
• It beats the best text-based RAG system by 18.1% on text QA because it preserves tables, charts, and layout that HTML parsers often drop (40%+ of content lost) or flatten, swinging accuracy by ~10 points.
• Ships as a Claude Code plugin that screenshots any URL, PDF, or local site and feeds pixels to a vision-language reader, with a single setup script and no extra backend.
• A separate described retrieval approach cuts corpus size 40x, reduces tokens per query 3x, and boosts vector-search relevance 2.3x through smarter chunking, dynamic indexing, and adaptive scoring.

Karpathy proposes replacing on-the-fly retrieval with an LLM-maintained markdown wiki that ingests sources, compiles structured pages, and self-updates through ingest, query, and lint cycles. This approach builds a persistent, compounding knowledge base without vectors or re-retrieval, though it currently lacks enterprise controls.

• Karpathy's "LLM Wiki" replaces vector-based RAG with an LLM that reads sources once and maintains a persistent, self-updating markdown wiki instead of re-retrieving chunked fragments per query.
• The system runs three operations—ingest (add new material), query (answer from the wiki, then save new answers back into it), and lint (scheduled scans for outdated info, broken links, and gaps)—so every interaction compounds the knowledge base rather than starting fresh.
• It directly targets two RAG failure modes: stateless retrieval that repeats embedding work every query, and chunking that destroys document context.
• The idea echoes Vannevar Bush's 1945 Memex concept of an associative, hyperlinked personal knowledge library, though it currently lacks enterprise-grade controls.

Organizations are increasingly faced with the decision of whether to implement Retrieval-Augmented Generation (RAG) or fine-tuning for their AI initiatives. RAG connects large language models to external databases, allowing access to real-time information, reducing inaccuracies, and enhancing security and traceability. However, implementing RAG comes with its own technical challenges that require careful planning and maintenance.

• RAG connects LLMs to external data sources so responses reflect real-time information rather than static training data
• Grounding responses in retrieved data cuts down on hallucinations and improves accuracy
• RAG offers added benefits like source traceability and better security for proprietary data
• Setting up RAG requires substantial infrastructure work—data pipelines, indexing, and ongoing maintenance for quality and scalability