GitHub - NikolayS/pgque: PgQue – Zero-bloat Postgres queue. One SQL file to install, pg_cron to tick.

Open original

PgQue is a pure SQL/PLpgSQL event queue for Postgres 14+ that delivers durable, Kafka-style streams without dead tuples or external daemons. It uses snapshot-based batching and TRUNCATE-driven table rotation for stable performance under sustained load. A pg_cron (or any scheduler) ticker drives 1–2 second end-to-end delivery.

  • Uses snapshot-based batching with TRUNCATE-driven table rotation instead of per-row UPDATE/DELETE with SKIP LOCKED, avoiding dead tuples, VACUUM pressure, and bloat under sustained load.
  • Pure SQL/PLpgSQL install (single file) with pg_cron ticking every second, so it runs on managed Postgres without C extensions or custom builds—unlike PgQ, PGMQ, River, Que, and pg-boss.
  • Offers Kafka-style shared event logs with independent per-consumer cursors, rather than one-job-one-worker task queue semantics.
  • Delivery latency is 1–2 seconds (tick interval plus polling), with microsecond function execution—fine for durability-focused use but not for sub-millisecond dispatch needs.

PgQue revives the Skype-era PgQ engine in pure SQL and PL/pgSQL for Postgres 14 and above, removing the need for C extensions or external daemons. You install it with a single SQL file and use pg_cron (or any scheduler) to run its ticker every second. Under the hood, PgQue uses snapshot-based batching and table rotation via TRUNCATE instead of per-row UPDATE/DELETE with SKIP LOCKED. That design avoids dead tuples, VACUUM pressure, index bloat and performance decay, even under sustained high throughput. Unlike task queues that hand each job to one worker, PgQue offers a shared event log with independent cursors per consumer. You push events, and each consumer reads from its own position, much like Kafka topics. Delivery latency defaults to 1–2 seconds (one tick interval plus polling) but per-call functions execute in microseconds. If you need sub-millisecond dispatch, pick a dedicated job queue; if you need stability under load without bloat, PgQue fits. A comparison table lines up PgQue against PgQ (the original C-extension version), PGMQ, River, Que and pg-boss. Only PgQue and PgQ use snapshot batching. PgQue is the only one that runs on managed Postgres without custom builds or shared libraries. It supports multiple consumers, built-in retry/backoff, dead-letter queues and a language-agnostic SQL API. Others rely on DELETE/UPDATE patterns that eventually bloat tables. To get started, load sql/pgque.sql inside a single transaction, then call pgque.start() to schedule ticker and maintenance jobs. If pg_cron runs in another database, you can configure cross-database cron calls. Finally, remember to disable or purge pg_cron logs to prevent unnecessary growth in cron.job_run_details.

Ask about this article