Click any tag below to further narrow down your results
+ fine tuning
(1)
+ hallucination
(1)
+ retrieval augmented generation
(1)
+ infrastructure
(1)
+ caveman compression
(1)
+ token optimization
(1)
+ context compression
(1)
+ prompting
(1)
+ jeff dean
(1)
+ ai agents
(1)
+ context engineering
(1)
+ orchestration
(1)
+ kimi k3
(1)
+ long context
(1)
+ context graph
(1)
Links
@0xMortyx on X
A data scientist ran a blind test pitting RAG against Kimi K3's full 1M-token context window on the same 12 questions. Long context scored perfectly on answer quality but cost about 16 times as much and ran roughly 3 times slower, so the post argues the right choice depends on corpus size and query volume.
- Long context answered all 12 questions perfectly, while RAG was near-perfect but weaker on completeness.
- Long context cost about 16 times as much and took roughly 3 times as long per question; at 12,000 queries the cost gap reaches about $3,800 versus $230.
- Skipping RAG is only backed by evidence when the corpus sits well under a fifth of the window; the test never covered corpora filling 60-80% of it.
- For tasks spanning many unconnected sources, the author points to Kimi K3's Agent Swarm and a connected context graph instead of either approach.
@sairahul1 on X
A post from @sairahul1 recommends a lecture by former Google Chief Scientist Jeff Dean that walks through the full AI engineering stack, from building LLMs from scratch to one person coordinating 100 agents. It also points to a linked course on production agent systems covering orchestration, RAG, evals, context engineering, and inference.
- The lecture covers the whole stack, from training LLMs from scratch up to coordinating large numbers of agents.
- The author says it suits both newcomers who haven't used a model and engineers already shipping agent systems.
- The linked course argues developers should learn how production agents work rather than take another prompt engineering course.
Caveman Compression strips predictable words like articles, fillers, and connectives to cut token count by 15–58% while preserving facts. It offers three methods—rule-based NLP, masked-LM, and LLM-based—to balance speed, cost, and compression ratio. Use it to pack more content into limited token budgets, especially for LLM reasoning and RAG contexts.
- Three compression modes trade off cost/speed for ratio: rule-based NLP (15-30%, <100ms, free), RoBERTa MLM (20-30%, 1-5s, free), and LLM-based via OpenAI API (40-58%, ~2s, needs API key)
- Benchmarks across 13 test cases show 100% fact preservation with ~40% average token reduction
- Compressed RAG contexts can store 2-3x more information, and agents reasoning directly in caveman format fit more chain-of-thought steps per context window
- Best suited for internal docs and agent reasoning, not for user-facing, legal, or emotional text where tone and grammar matter
Organizations are increasingly faced with the decision of whether to implement Retrieval-Augmented Generation (RAG) or fine-tuning for their AI initiatives. RAG connects large language models to external databases, allowing access to real-time information, reducing inaccuracies, and enhancing security and traceability. However, implementing RAG comes with its own technical challenges that require careful planning and maintenance.
- RAG connects LLMs to external data sources so responses reflect real-time information rather than static training data
- Grounding responses in retrieved data cuts down on hallucinations and improves accuracy
- RAG offers added benefits like source traceability and better security for proprietary data
- Setting up RAG requires substantial infrastructure work—data pipelines, indexing, and ongoing maintenance for quality and scalability