Click any tag below to further narrow down your results
+ llm
(4)
+ retrieval
(4)
+ retrieval augmented generation
(3)
+ context engineering
(2)
+ agents
(2)
+ long context
(2)
+ token optimization
(1)
+ context compression
(1)
+ prompting
(1)
+ mcp server
(1)
+ karpathy
(1)
+ knowledge base
(1)
+ caveman compression
(1)
+ ai
(1)
+ recursive language models
(1)
Links
This is a sales page for Decodo's Web Data API, which offers a free starter tier with $5 in credit and up to 10K requests a month across four endpoints (/scrape, /search, /map, /crawl). It pitches the API as a single-key tool for pulling clean, LLM-ready web data for AI agents and pipelines.
- The free plan gives $5 credit and up to 10K requests per month with no card required, covering all four endpoints.
- Output comes as Markdown or JSON with nav bars and ads stripped, aimed at RAG pipelines and LLM context windows.
- It ships an official MCP Server plus LangChain and LlamaIndex integrations, and claims a 99.99% success rate with pricing from $0.15 per 1K requests.
STAIR is a retrieval system that feeds an LLM a book's table of contents so it can store and fetch information from its parameters, rather than chunking text by length. The authors also release SearchTome, a benchmark built from 18 books across six domains.
- Standard retrievers split long documents into fixed-length chunks and discard global structure like the table of contents; STAIR keeps that structure.
- On SearchTome, STAIR reaches 82.6% Recall@1, versus 76.9% for DSI, 68.7% for DPR, and 59.5% for BM25; the authors report the DSI gap as statistically significant.
- The authors claim a hallucination rate under 0.05% for the generative retrieval setup and say it generalizes to cases with very few training samples.
This is the second part of an investigation into whether OpenEvidence, a medical AI company valued at $15B, has technology and content worth that price. The author finds the proprietary AI, content ownership, and accuracy claims all look weak under scrutiny.
- OpenEvidence doesn't own its content; publishers can pull their material at any time, which undercuts the moat a buyer would be paying for.
- Accuracy claims collapse outside exam settings: the company's near-perfect USMLE scores drop to about 27% on real-world clinical cases.
- Ad revenue projections ($400M to $500M in 2026) still put the $15B valuation at 30x to 37.5x a single year of sales, and the company's own CEO now pitches drug discovery as the real prize.
This is an AWS Marketplace tutorial for a retrieval-augmented generation setup: documents land in S3, a Lambda function chunks them and writes embeddings (via Amazon Titan on Bedrock) into an Elastic vector index, and API Gateway exposes a chat agent over that data. It ships as a Terraform module that automates the whole pipeline.
- The pipeline is fully event-driven: an S3 upload triggers a containerized Lambda that loads, splits, embeds, and stores the documents in Elastic with no separate ETL infrastructure.
- Elastic is positioned as one store for both full-text and vector search, connected privately to AWS via PrivateLink, with a free trial and a Quick Launch deployment.
- Amazon Titan models on Bedrock handle the embeddings and answer generation, with API Gateway providing the user-facing endpoint.
David Breunig's talk on how context engineering emerged after the long-context push of 2024, why bigger context windows introduced new failure modes, and the six patterns builders now use to manage what goes into a model's context. It's a useful map of where the practice has been and where it's headed.
- Million-token context windows didn't solve the problem. Long contexts add room for errors as well as information, leading to failure modes like context poisoning, distraction, confusion, and clash.
- Gemini 2.5 playing Pokémon shows poisoning in practice: a fact baked into its weights (the tea bribe from the older game) got stuck in its context and cost it hours. Performance also degraded past roughly 100,000 tokens, with the agent repeating old actions instead of planning new ones.
- Breunig's research cites shows instruction-following drops as bad examples pile up in context, affecting every model tested, including smaller ones that fall below half adherence after a few contradictions. The six-tool toolbox (augmentation, loadout, quarantine, pruning, offloading, summarization) has stayed stable as these problems emerged.
The author tested whether agentic retrieval-augmented generation, where an LLM agent rewrites queries and decides what to fetch, actually finds better evidence than plain search and reranking. The whole setup is reproducible from a GitHub repo, with 298 reviewed questions and saved results.
- The test pits agentic RAG against plain search and a reranker on 298 questions, with Claude models writing the questions and running the agent.
- Results are reproducible: the code, questions, and saved outputs for every run are published, so you can check the numbers yourself.
- Reciprocal-rank fusion is the core merge step, combining several ranked lists of retrieved documents into one.
PyDebeziumAI is a Python library that streams row-level changes from Postgres through Debezium into a vector store like Chroma, PGVector, or Milvus, so LangChain and LangGraph retrieval stays current instead of depending on periodic batch reloads. It runs an embedded Debezium JVM, so it needs Java 17+.
- Captures every INSERT, UPDATE, and DELETE from the database write-ahead log and pushes it into the vector store as it happens, avoiding stale RAG results.
- Uses deterministic primary-key document IDs and idempotent upserts, so redelivered events after a restart don't create duplicates.
- Works offline with local sentence-transformers embeddings, with no API key required, and supports Chroma, PGVector, and Milvus backends.
- The README cites roughly 280–500ms total indexing time, though that figure depends on the embedding model and vector database.
@0xMortyx on X
A data scientist ran a blind test pitting RAG against Kimi K3's full 1M-token context window on the same 12 questions. Long context scored perfectly on answer quality but cost about 16 times as much and ran roughly 3 times slower, so the post argues the right choice depends on corpus size and query volume.
- Long context answered all 12 questions perfectly, while RAG was near-perfect but weaker on completeness.
- Long context cost about 16 times as much and took roughly 3 times as long per question; at 12,000 queries the cost gap reaches about $3,800 versus $230.
- Skipping RAG is only backed by evidence when the corpus sits well under a fifth of the window; the test never covered corpora filling 60-80% of it.
- For tasks spanning many unconnected sources, the author points to Kimi K3's Agent Swarm and a connected context graph instead of either approach.
@0xMortyx on X
A 90-minute Google Cloud workshop walks through building production AI agents, moving from basic retrieval-augmented generation (RAG) to Graph RAG, memory, and multi-agent orchestration. The post shares a timestamped chapter guide and links to a written roadmap covering the same progression.
- The workshop is organized as a progression: RAG, then Graph RAG, then memory, then multimodal agent graphs.
- Hands-on sections cover setting up a production agent stack, Graph RAG with semantic search, extracting graph context, and orchestrating specialized agents.
- The author frames it as one of the clearer hands-on examples of teams moving beyond plain RAG toward graph-based context.
@sairahul1 on X
A post from @sairahul1 recommends a lecture by former Google Chief Scientist Jeff Dean that walks through the full AI engineering stack, from building LLMs from scratch to one person coordinating 100 agents. It also points to a linked course on production agent systems covering orchestration, RAG, evals, context engineering, and inference.
- The lecture covers the whole stack, from training LLMs from scratch up to coordinating large numbers of agents.
- The author says it suits both newcomers who haven't used a model and engineers already shipping agent systems.
- The linked course argues developers should learn how production agents work rather than take another prompt engineering course.
Open Executive is an open-source system that uses Claude to simulate a cohesive executive team—eight specialized AI agents (CFO, CSO, CMO, etc.) that maintain context across sessions and proactively surface follow-ups. It's designed to run locally or on Fly.io with integrations for Slack, Discord, email, and other platforms.
- Eight specialist agents route through a single orchestrator to synthesize responses in one consistent executive voice, with built-in MBA knowledge layered alongside your uploaded company documents via RAG.
- Episodic memory in SQLite tracks past decisions and initiatives across sessions, and a scheduler proactively surfaces due actions—but the API must run single-instance only to avoid double-firing scheduled tasks.
- Deploys to Fly.io with separate dev and qa environments, supports document upload (pitch decks, financials, strategy docs), and offers multiple chat interfaces (web UI, Slack, Discord, email, Telegram, CLI).
After Andrej Karpathy outlined the idea of “LLM Knowledge Bases” on April 3, 2026, developers built Graphify in under 48 hours. Graphify turns any folder of code, docs, and images into an interlinked markdown wiki you can query without vector databases or complex setup. It persists across sessions, builds backlinks and concept maps, and cuts future query costs.
- Karpathy's April 3, 2026 pitch for "LLM Knowledge Bases" proposed replacing RAG's fetch-at-query-time approach with an LLM-generated, persistent markdown wiki.
- Developers built Graphify from that idea in under 48 hours, and it racked up hundreds of GitHub stars within hours of posting.
- Graphify converts any folder (code, docs, images) into an interlinked markdown wiki—summaries, backlinks, concept maps, index—with one command, no vector database or retrieval pipeline needed.
- Because the wiki is plain markdown that compounds with each update, it's transparent, Git-versionable, and cuts future query costs compared to traditional embedding-based retrieval.
@TheGlobalMinima on X
The author says a site called paperswithcode.co, which tracks the most cited papers in modern generative AI, is the best way to learn the field. The pitch is to read the top 10 most cited papers at a pace of one or two per week, then reread, reimplement, and write up what you learned.
- Covers language modelling, inference optimisation, RL, system scaling, agentic AI, RAG, memory, and benchmarking in one place, which the author says is rare in nearly five years of generative AI.
- Argues these subfields overlap in practice: agents need system scaling, memory needs inference optimisation, and RL depends on understanding environments and benchmarks.
- Recommends a slow, repeated reading routine: one or two papers per week, reread, break down, code it, and write it back.
The article shows how packing specialized, retrieved knowledge into smaller AI agents can match or beat huge frontier models. It explains a structure—raw source extractions, concept entries, theses and a startup primer—plus embedding‐powered retrieval to feed just the right context at query time.
- Structured retrieval (10,000 pages → 381 concept docs + 54 theses via hybrid BM25/semantic search) let smaller models match or beat frontier models on specialized tasks.
- The same pipeline replicated across a dozen domains, from finance to rare medical research to corporate policy.
- A locally-run Qwen model with this "knowledge agent" harness performed comparably to Claude Opus, at zero cloud cost.
- Embedding costs were trivial (under a dollar for thousands of documents using BGE-M3 or OpenAI's text-embedding-3-small) before moving entirely to local hardware.
@Oliviacoder1 on X
Three MIT CSAIL researchers (Zhang, Kraska, Khattab) propose Recursive Language Models, which store a long document as a variable outside the model's context and have it search and read only the relevant parts, spawning sub-models to handle sections in parallel. The post claims this handles inputs about 100 times larger than a model's native window with better accuracy at similar cost, and the code is open source on GitHub.
- Standard frontier models score near zero on the hardest long-context reasoning benchmarks, and GPT-5 couldn't solve even 10% of problems requiring tracking code history beyond 75,000 tokens.
- RLMs keep the full document intact and queryable instead of summarizing or chunking it, and reportedly handle inputs up to 10 million tokens with double-digit gains over alternatives.
- The cost per query is reported as comparable to or cheaper than large-context calls, and the code is free on GitHub as a drop-in replacement for existing LLM API calls.
Karpathy proposes replacing on-the-fly retrieval with an LLM-maintained markdown wiki that ingests sources, compiles structured pages, and self-updates through ingest, query, and lint cycles. This approach builds a persistent, compounding knowledge base without vectors or re-retrieval, though it currently lacks enterprise controls.
- Karpathy's "LLM Wiki" replaces vector-based RAG with an LLM that reads sources once and maintains a persistent, self-updating markdown wiki instead of re-retrieving chunked fragments per query.
- The system runs three operations—ingest (add new material), query (answer from the wiki, then save new answers back into it), and lint (scheduled scans for outdated info, broken links, and gaps)—so every interaction compounds the knowledge base rather than starting fresh.
- It directly targets two RAG failure modes: stateless retrieval that repeats embedding work every query, and chunking that destroys document context.
- The idea echoes Vannevar Bush's 1945 Memex concept of an associative, hyperlinked personal knowledge library, though it currently lacks enterprise-grade controls.
Caveman Compression strips predictable words like articles, fillers, and connectives to cut token count by 15–58% while preserving facts. It offers three methods—rule-based NLP, masked-LM, and LLM-based—to balance speed, cost, and compression ratio. Use it to pack more content into limited token budgets, especially for LLM reasoning and RAG contexts.
- Three compression modes trade off cost/speed for ratio: rule-based NLP (15-30%, <100ms, free), RoBERTa MLM (20-30%, 1-5s, free), and LLM-based via OpenAI API (40-58%, ~2s, needs API key)
- Benchmarks across 13 test cases show 100% fact preservation with ~40% average token reduction
- Compressed RAG contexts can store 2-3x more information, and agents reasoning directly in caveman format fit more chain-of-thought steps per context window
- Best suited for internal docs and agent reasoning, not for user-facing, legal, or emotional text where tone and grammar matter
Organizations are increasingly faced with the decision of whether to implement Retrieval-Augmented Generation (RAG) or fine-tuning for their AI initiatives. RAG connects large language models to external databases, allowing access to real-time information, reducing inaccuracies, and enhancing security and traceability. However, implementing RAG comes with its own technical challenges that require careful planning and maintenance.
- RAG connects LLMs to external data sources so responses reflect real-time information rather than static training data
- Grounding responses in retrieved data cuts down on hallucinations and improves accuracy
- RAG offers added benefits like source traceability and better security for proprietary data
- Setting up RAG requires substantial infrastructure work—data pipelines, indexing, and ongoing maintenance for quality and scalability