Build1 distinct publisher3 min readUpdated
A side project swapped RabbitMQ, Redis and Elasticsearch for SKIP LOCKED, an unlogged table and a GIN index. The queue arithmetic shows where that choice stops working.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
A developer running a handful of content agents needed three things beyond his main database: a job queue, a cache for repeated lookups, and search over article text [1]. Instead of adding services he put all three in Postgres, and that matters because at small scale the marginal service is not a feature but a Docker image, a distinct failure mode and its own 3 AM pager behaviour [4][3].
The default stack he reached for first was RabbitMQ or Redis Streams for the queue, Redis for the cache, and Elasticsearch or Meilisearch for search [2]. Counting processes changed the decision: one Spring Boot app plus three more moving parts is four things to run where one would do [3][24]. What replaced them was a table claimed with SKIP LOCKED, an unlogged table for the cache, and a tsvector column with a GIN index for search, all inside one schema with one backup strategy and one connection pool [4][5].
The queue is the part he works through in code, and it is the least exotic. The naive approach, SELECT followed by UPDATE WHERE status = 'pending', races between workers [7]. Postgres has shipped the fix since 9.5: SELECT ... FOR UPDATE SKIP LOCKED, which skips rows another worker already holds [6]. The claim is one statement, an UPDATE whose subquery does FOR UPDATE SKIP LOCKED with a LIMIT and a RETURNING * on the outside, so concurrent workers get disjoint batches without a second round trip [8]. No advisory locks, no leader election, no broker [9]. The docs cover it under row-level locking, and Crunchy Data has a deep dive on native Postgres queuing [20]. Jobs survive restarts because they live in a table that is already in the backup [11].
Now the arithmetic, because this is where honest limits live. The poller in the post is a scheduled method firing every 2000 ms and claiming 5 jobs [10]. That is 150 jobs per minute per worker [21], with a worst case wait of 2 seconds before a ready job is picked up and about 1 second on average [23]. The author's own stated breaking point is millions of jobs per minute, streaming semantics, or long retention for event replay, at which point you want a real broker [14]. Reaching one million jobs per minute at that batch and interval would take roughly 6,700 workers [22], which is another way of saying the gap between this pattern and Kafka is several orders of magnitude wide, not a close call. What you are accepting in exchange is at-least-once delivery with polling latency rather than push, and no replayable ordered log [13]. The thing you skip is bootstrapping a broker, writing consumers, configuring dead-letter exchanges and understanding prefetch [12].
The precedent cited is real but uneven: Bauer's essay "PostgreSQL for Everything" was on the Hacker News front page this week, Contentful rebuilt full-text search on Postgres rather than a separate cluster, Instacart built search infrastructure on Postgres, and the Guardian moved parts of its platform off MongoDB onto Postgres [15][16][17][18]. The author is explicit that he is one developer in Dhaka with a VPS and has run these patterns in his own projects, not at giant scale [19].
Watch your own arrival rate against the poll interval. If depth grows between polls, tune batch and interval first; that is a config change, not a new pager rotation. And note that the queue pattern here comes with a stated failure boundary [13][14] while the unlogged cache table and the GIN search index are named rather than stress-tested [4], so treat those two as the parts you still have to prove.
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
A developer building AI agent infrastructure for agents that research, draft and publish content needed three things beyond the main database: a job queue, a cache for repeated lookups, and search over article text.
His first instinct was RabbitMQ or Redis Streams for the queue, Redis for the cache, and Elasticsearch or Meilisearch for search.
He counted the services: one Spring Boot app plus three more moving parts, each with its own Docker image, its own failure modes and its own 3 AM pager behavior.
He instead did all three jobs in Postgres: a table with SKIP LOCKED for the queue, an unlogged table for the cache, and a tsvector column with a GIN index for search.
The resulting system is one schema, one backup strategy and one connection pool.
The author describes the result as a durable job queue in about forty lines, with jobs surviving restarts because they live in a table you already back up.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
One self-reported implementation with code, no measurements
The cluster is a single practitioner post. Its mechanism claims are verifiable and well-anchored — SKIP LOCKED since 9.5, disjoint batches via a subquery, UNLOGGED tables not being crash-safe — and working code is shown, plus pointers to Postgres docs and a Crunchy Data deep dive. But there are no benchmarks, no latency or throughput measurements, no second publisher, and the third-party precedents and Hacker News essay are asserted without citation. The only quantitative boundary in the cluster comes from arithmetic on the published poller config, not from measurement.
One personal side project; precedents unverified
Directly evidenced adoption is a single solo developer's side project on one VPS, running roughly a week for the cache path. The only other adoption signal is a three-company list offered without citation. There is no release, deployment, pricing or usage disclosure from any vendor or organisation in the cluster, so measured adoption stays near the floor while remaining non-zero because the author reports first-hand production-style use of all three patterns.
Deletion framing outruns the measured record, body hedges
The framing — three services you can delete, a durable queue in about forty lines, one Postgres for everything — is broader than the record supports: one side project, no measurements, unverified enterprise precedents. The overstatement is modest rather than severe because the author self-discloses limited scale, names at-least-once delivery and polling latency as real constraints, and directs high-throughput or replay workloads to a real broker. The gap is sharpest where numbers are absent: the published poller implies roughly 150 jobs per minute per worker and a claim delay of up to two seconds, and the 'forty lines' figure excludes the reaper the author says completes the story.
Practitioner-credibility incentive, no disclosed vendor stake
The post is a personal developer-platform article about the author's own side project, so the operative incentive is reputational and audience-building: a strong simplification narrative performs well and rides an active Hacker News discussion the author cites. Nothing in the cluster indicates vendor sponsorship, commercial affiliation with Postgres services, or product being sold, and the author volunteers scale limitations that cut against his own framing, which lowers the distortion reading.
Mechanisms trustworthy, scale and precedent claims weak
Confidence is split. The database mechanics are highly credible: SKIP LOCKED semantics, single-statement claiming with RETURNING, and UNLOGGED-table trade-offs are standard, documented Postgres behaviour shown in working code, and the derived arithmetic follows directly from the published configuration. Confidence drops sharply for anything beyond one machine — no measurements, one publisher, uncited third-party adoption, and no treatment of vacuum, bloat or pool contention — so the overall reading lands just under the midpoint.
build
A RAG stack lived seven hours before a hosted embedding endpoint returned 4041 distinct publisher
build
One Mutable vat_id Column Is An Audit Failure With A Delay Fuse1 distinct publisher
build
The stopping problem: an LLM rewrite loop that converged on code javac rejected1 distinct publisher
build
A 27B Apache-2.0 model in 17GB makes local inference a wiring decision, not a demo1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 20, 2026