GitHub - timescale/pg_vectorsearch: High-performance vector search for PostgreSQL
Short summary
pg_vectorsearch is an alpha PostgreSQL extension that indexes vectors using PRISM, a hierarchical partitioning method with quantization. It outperforms pgvector's IVFFlat and HNSW indexes on speed and recall, even while using a quarter of the memory on 100-million-vector datasets.
- PRISM uses a hierarchical centroid tree, RaBitQ quantization with error bounds, and SIMD-optimized distance computation to achieve faster queries and index builds than existing PostgreSQL vector indexes.
- Works with both native vec32/vec16 column types and existing pgvector vector/halfvec columns without requiring table rewrites or query changes.
- Supports dynamic updates (inserts, updates, deletes) with a prism_rebalance procedure to split oversized posting lists, and requires PostgreSQL 18+ with a C11 compiler.
Full summary
pg_vectorsearch is a PostgreSQL extension that indexes vectors using PRISM, a new access method designed to beat the performance of pgvector's existing IVFFlat and HNSW indexes. The core trick is hierarchical partitioning via a centroid tree, combined with RaBitQ quantization—a scheme that encodes vectors with error bounds tight enough to rule out candidates without checking them. On a 100-million-vector dataset, PRISM outpaces HNSW on both query speed and index build time while using a quarter of the memory. The index lives in the PostgreSQL buffer cache and uses async I/O to scan posting lists, then applies two-stage filtering: first a quick estimate with error bounds to eliminate impossibilities, then full-precision reranking on survivors. The extension is written in C and uses SIMD (AVX2, AVX512, NEON) to accelerate distance math.
You can use PRISM two ways depending on your setup. Start from scratch with pg_vectorsearch's own vec32 (float32) or vec16 (float16) types, or drop it into an existing pgvector deployment—it indexes vector and halfvec columns directly without rewriting them. Both paths use the same operators (<->, <#>, <=>) for L2, inner product, and cosine distance. The extension is currently alpha, so upgrades between pre-releases aren't guaranteed and you may need to drop and recreate indexes. It requires PostgreSQL 18+, a C11 compiler, and Meson to build.
The main tuning knob is prism.nprobe, which controls how many clusters to search (default ~0.95 recall). Inserts append to posting lists and can cause imbalance; the prism_rebalance procedure splits oversized lists in place. The docs cover architecture, tuning parameters, maintenance procedures, and build options. The work draws on published research: RaBitQ (SIGMOD 2024), SPFresh (SOSP 2023), and SPANN (NeurIPS 2021). If you're already running pgvector with HNSW indexes, you can build a PRISM index alongside it, verify it's faster with EXPLAIN, then drop the old one.