The first search engine with fully vectorized query execution

Open original

turbopuffer is rebuilding its query engine so every query runs through a batch-oriented vectorized pipeline, borrowing an approach usually found in analytical databases like DuckDB. The team argues search engines have skipped this because they ran narrow ranking loops, but customer queries now combine filters, boosts, and aggregations that make operator dispatch a real cost.

  • Volcano-style row-at-a-time execution wastes CPU on per-row function calls and dispatch; vectorized engines process cache-friendly batches to cut that overhead.
  • Traditional search algorithms like block-max WAND and HNSW resist vectorization because each similarity result decides what to examine next, creating data dependencies the CPU can't predict.
  • turbopuffer v1 had one similarity operator, but customers now stack filters, recency decay, aggregations, and custom scoring, so the query plans are heterogeneous enough to justify the switch.

Turbopuffer, a search engine, plans to run every query through a fully vectorized execution pipeline when its v3 ships to production later in 2026. The team says this would make it the first search engine to do so, an architecture more typical of analytical databases like DuckDB. The motivation is hardware: modern CPUs reward predictable, batch-oriented access patterns, and the post argues that search is moving beyond narrow top-k retrieval, so the engine needs to handle a wider range of query shapes well. The article explains the problem with the older approach first. The Volcano model, used by most traditional databases, chains generic operators like scan, filter, and aggregate, pulling one row at a time through each. Its weakness is per-row overhead: every row at every operator pays for a function call, dynamic dispatch, and interpretation, so the CPU spends more of its time deciding what to do next than doing the work. The post walks through a concrete example, summing prices under 32 from a small product table, to show how many tiny steps that involves. The two known fixes are compiled engines, which generate specialized code per query (Hyper is the cited example), and vectorized engines, which keep the generic operators but process cache-friendly batches of rows. Turbopuffer is taking the vectorized route, which avoids the cost of generating code per query while still amortizing dispatch overhead across many rows at once. What's less settled from this post alone is how the gains will show up in practice. The article makes the architectural case clearly but gives no benchmark numbers, so the claim of "very strong performance on many types of queries" is unverified here. It also doesn't say which query types benefit most or where batching might hurt, such as highly selective queries that return few rows. Readers interested in whether this matters for their own workloads should wait for v3 numbers, or check the reference the post points to, "Everything You Always Wanted to Know About Compiled and Vectorized Queries But Were Afraid to Ask," for the underlying trade-offs.

Ask about this article