Introducing TIN: full-text search for Postgres â PlanetScale

Open original

PlanetScale released TIN, a new full-text search index for Postgres that handles boolean queries, fuzzy matching, BM25 ranking, and concurrent writes—capabilities existing Postgres search tools lack. In benchmarks against competitors, TIN processed 25-541x more queries per second with dramatically lower latency.

  • TIN supports boolean expressions, phrase queries, fuzzy/wildcard/regex matching, BM25 scoring, and handles concurrent updates—a combination no existing Postgres full-text index provides
  • Benchmark results show TIN handles 25x more queries/second than ParadeDB and 541x more than Postgres GIN on mixed workloads, with p99 latencies 26-1,356x lower
  • With concurrent writes, TIN completed 270,279 updates over 10 minutes while ParadeDB managed 185,584 and pg_textsearch only 735, showing it doesn't sacrifice write performance for read speed

PlanetScale has released TIN, a full-text search extension for Postgres that handles queries existing options can't manage well. The tool supports boolean expressions, phrase queries, fuzzy matching, regex, and BM25 scoring while maintaining transaction visibility and working correctly with joins, concurrent updates, and replication. Three other text-search indexes already exist for Postgres—ParadeDB, pg_textsearch, and the built-in GIN index—but none handle all these requirements simultaneously. TIN does, and the benchmarks suggest it's substantially faster. The performance gaps are significant. On a 150-million-document Stack Exchange corpus, TIN handled 25 times more queries per second than ParadeDB for mixed workloads and achieved p99 latencies 26 times lower. For conjunction and phrase queries, it processed 10 times more throughput than ParadeDB and 541 times more than GIN. The real standout came during concurrent writes: TIN completed 270,279 updates over ten minutes while ParadeDB managed 185,584 and pg_textsearch only 735. Index build time was 8 minutes 10 seconds compared to 19 minutes for ParadeDB and over two hours for GIN. The benchmarks used realistic conditions: an 85 GB corpus with 1,719 queries generated from Stack Exchange data, tested on an AWS i7i.8xlarge instance with 8 vCPUs and 32 GB of RAM. The setup ensured the index wouldn't simply live in memory, forcing realistic disk I/O patterns. ParadeDB was the only other engine that completed all tests; GIN and pg_textsearch failed on various workloads due to memory constraints or missing functionality. The results show TIN isn't just marginally better—it's orders of magnitude faster at the specific problem of handling full-text search at scale with concurrent data changes.

Ask about this article