RIP, vector database
Short summary
Turbopuffer is rebuilding its database engine to stop treating vector indexes as the primary key. The company found that organizing everything around ANN addresses works great for vector search but cripples performance for filtering, full-text search, and aggregations—so they're moving to a new primary index that treats vectors as just another secondary index.
- The current architecture forces document duplication when storing multi-vector representations and cascades full document rewrites whenever the ANN index rebalances, creating massive write amplification that's hitting diminishing returns on throughput tuning.
- Non-vector query plans are constrained to tiny block sizes (100–200 docs) by the ANN index, while modern query engines need thousands of documents per block to vectorize efficiently—full-text search got 10x smaller and 20x faster just by switching to 256-doc blocks.
- Turbopuffer v3 passes all CI tests but shows significant performance regressions; the team is now in the optimization phase and plans to publish benchmarks as they tune the new architecture.
Full summary
Turbopuffer is overhauling its storage architecture because the original design—built around a hierarchical clustering vector index (SPFresh) as the primary key—has become a bottleneck. The company started as a specialized serverless vector database using cheap object storage with cached NVMe SSDs, and that architecture worked brilliantly for pure vector search at massive scale (100B+ vectors, 200ms p99 latency). But over time, customers demanded features beyond ANN: attribute filtering, BM25 full-text search, aggregations, regex, fuzzy matching, sparse vectors, and ordering. Each new query type got bolted on as a secondary index pointing back to the ANN addresses, which meant the entire system remained constrained by the vector clustering logic.
Three specific problems emerged from keeping the ANN index as the primary key. First, storage amplification: when a document has multiple vector representations (like nested documents or late interaction models), all non-vector content gets duplicated for each vector. Second, write amplification: whenever SPFresh rebalances vectors to maintain clustering quality, it cascades into moving the full document contents plus all attribute and FTS indexes that reference it—updating one vector can shuffle hundreds of attributes around. Third, block size constraints: modern query engines like DuckDB and ClickHouse work on blocks of thousands of rows for CPU efficiency and SIMD, but turbopuffer's non-vector operations are stuck at 100–200 document blocks because that's the ANN cluster size. Full-text search v2 proved this matters—moving from 1.5 average postings per block to 256 made indexes 10x smaller and queries 20x faster.
Turbopuffer v3 solves this by making the ANN index just another secondary index instead of the primary key. This isn't a minor tweak; it's a fundamental redesign of how documents and indexes are laid out, written, and compacted. The team hit a major milestone recently with 100% CI pass rate, but the new version currently runs slower than production. They've focused on correctness and foundational design first, and now they're entering the performance tuning phase. The company is documenting this work publicly as they grind through the optimization process.