GitHub - wilpel/caveman-compression: Caveman Compression is a semantic compression method for LLM contexts. It removes predictable grammar while preserving the unpredictable, factual content that defines meaning.
Short summary
Caveman Compression strips predictable words like articles, fillers, and connectives to cut token count by 15–58% while preserving facts. It offers three methods—rule-based NLP, masked-LM, and LLM-based—to balance speed, cost, and compression ratio. Use it to pack more content into limited token budgets, especially for LLM reasoning and RAG contexts.
- Three compression modes trade off cost/speed for ratio: rule-based NLP (15-30%, <100ms, free), RoBERTa MLM (20-30%, 1-5s, free), and LLM-based via OpenAI API (40-58%, ~2s, needs API key)
- Benchmarks across 13 test cases show 100% fact preservation with ~40% average token reduction
- Compressed RAG contexts can store 2-3x more information, and agents reasoning directly in caveman format fit more chain-of-thought steps per context window
- Best suited for internal docs and agent reasoning, not for user-facing, legal, or emotional text where tone and grammar matter
Full summary
Caveman Compression strips predictable words—articles, conjunctions, passive markers, filler terms—to cut 15–58% of tokens while keeping all technical facts intact. You feed verbose text into one of three modes: rule-based NLP (free, offline, 15–30% reduction in <100 ms), MLM-based using RoBERTa (free, offline, 20–30% reduction in 1–5 s), or LLM-based via OpenAI API (requires key, 40–58% reduction in ~2 s). Each mode outputs staccato “caveman” sentences like “Network router forwards data packets” instead of full grammar, trusting your LLM to refill missing words when needed.
Benchmarks show 100% of facts preserved across 13 test cases and average token cuts of 40%. Examples span API docs, system prompts, resumes and step-by-step instructions. A compressed RAG context can store 2–3× more information, and agents think faster when they operate directly on caveman format—chain-of-thought proofs fit more steps in the same window.
Setup is straightforward: clone GitHub repo, create Python venv, install requirements (`requirements.txt`, `requirements-nlp.txt` or `requirements-mlm.txt`), download spaCy models, then run `caveman_compress_*.py compress` or decompress. SPEC.md lists full removal rules. Examples, benchmarks, and prompt templates live in their respective folders.
Use caveman compression for internal docs, agent reasoning blocks or any token-budgeted workflow. Avoid it in user-facing, marketing, legal or emotional text where full grammar and tone matter. MIT-licensed, maintained by William Peltomäki.