More on the topic…
PlanetScale has released TIN, a full-text search extension for Postgres that handles queries existing options can't manage well. The tool supports boolean expressions, phrase queries, fuzzy matching, regex, and BM25 scoring while maintaining transaction visibility and working correctly with joins, concurrent updates, and replication. Three other text-search indexes already exist for Postgres—ParadeDB, pg_textsearch, and the built-in GIN index—but none handle all these requirements simultaneously. TIN does, and the benchmarks suggest it's substantially faster.
The performance gaps are significant. On a 150-million-document Stack Exchange corpus, TIN handled 25 times more queries per second than ParadeDB for mixed workloads and achieved p99 latencies 26 times lower. For conjunction and phrase queries, it processed 10 times more throughput than ParadeDB and 541 times more than GIN. The real standout came during concurrent writes: TIN completed 270,279 updates over ten minutes while ParadeDB managed 185,584 and pg_textsearch only 735. Index build time was 8 minutes 10 seconds compared to 19 minutes for ParadeDB and over two hours for GIN.
The benchmarks used realistic conditions: an 85 GB corpus with 1,719 queries generated from Stack Exchange data, tested on an AWS i7i.8xlarge instance with 8 vCPUs and 32 GB of RAM. The setup ensured the index wouldn't simply live in memory, forcing realistic disk I/O patterns. ParadeDB was the only other engine that completed all tests; GIN and pg_textsearch failed on various workloads due to memory constraints or missing functionality. The results show TIN isn't just marginally better—it's orders of magnitude faster at the specific problem of handling full-text search at scale with concurrent data changes.
Questions about this article
No questions yet.