Skip to content

Build1 publisher3 min readPublished

A seven-second job on a five-second setInterval overlaps its own next run by two seconds

A dev.to teardown makes the case for event-triggered Node workers with a slow poll behind them, and the idle-cost numbers it publishes come to about 138,000 empty queries a day across eight instances.

The Engineer · Build desk

Illustration accompanying A seven-second job on a five-second setInterval overlaps its own next run by two seconds

What happened

  • setInterval schedules by start time, so in the post's example a run that takes seven seconds on a 5,000 ms timer begins again two seconds before the previous run has finished.
  • Postgres NOTIFY and Redis pub/sub are fire-and-forget, so a listener disconnected by a deploy, a network blip or a failover loses the notification and the job sits in the table with nothing retrying it.
  • The recommended shape keeps the notification handler for latency and adds a 60-second poll as the safety net, a twelfth of the five-second idle cost, finding nothing on a healthy day.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • cost Cutting the poll from five seconds to a minute takes roughly 126,720 empty queries a day off an eight-instance deployment, and the bill lands on whoever pays per query on a metered database or per wake-up on battery.
  • constraint Adopting events means the team now owns delivery, and because NOTIFY does not retry, the queue table plus a running poll is the only thing standing between a mid-deploy notification and a job nobody ever picks up.
  • capability The listener buys millisecond pickup for work that previously waited up to a full poll interval, which makes user-triggered background jobs viable without shortening the timer.

The unbounded part of that overlap claim needs a condition attached. At seven seconds of work per five-second tick, the number of runs in flight settles near 1.4 [1]. It stays there for as long as each run still takes seven seconds. It climbs only when the extra concurrency slows each run down, and the post spells out why: every overlapping run is holding connections while it works, until the pool is exhausted [3].

The rewrite is a while loop that awaits the work, then awaits a five-second timer, so the gap starts when the run ends and a slow run delays the next one [4]. The try/catch inside that loop is the other half. Without it, one thrown error ends the worker silently and nothing processes jobs until somebody notices [5].

Twelve polls a minute is 17,280 a day [6]. Eight instances of that is 138,240, which the post rounds to about 138,000 queries a day that find nothing, plus the connection each one holds and a wake-up that stops the runtime idling [2][7]. Move the poll to once a minute and each instance does 1,440, or 11,520 across the eight, taking roughly 126,720 empty queries a day off the total [3]. The post is explicit about who notices: invisible on a laptop, a real line item on a metered database or on anything running on battery [8].

LISTEN/NOTIFY buys latency in milliseconds and an idle system that does nothing at all [10], and the cost is delivery. Postgres NOTIFY is fire-and-forget, so a listener disconnected by a deploy, a network blip or a failover loses the notification, nothing retries it, and the row sits in the table; Redis pub/sub behaves the same way [11]. The slow poll is the only thing that will ever find that row.

On a healthy day the 60-second poll finds nothing, which the post calls exactly what you want it to find [13]. A dead listener therefore looks like a healthy one from the outside. Counting rows claimed by the notification path and the poll path separately tells them apart.

Poll-only remains correct where a minute of delay is irrelevant, and it needs no extra infrastructure [18]. If that describes your jobs, stay with the interval and skip the rest of this.

Both paths can fire at once, so the drain function needs an in-flight flag [14]. It should keep claiming batches until a claim returns zero rows, because a notification means there is work, not that there is exactly one job [15]. Past one instance, claiming has to be atomic or two workers process the same job, and FOR UPDATE SKIP LOCKED lets each worker take rows nobody else has locked instead of queueing behind them [16]. Then a sweep that returns jobs stuck in running past a timeout, because a worker will be killed mid-job eventually and that row has to come back [17].

What to watch

  • A broker that acknowledges deliveries changes the delivery argument; the fire-and-forget warning is specific to Postgres NOTIFY and Redis pub/sub.
  • The saving is a query count, not a sum of money, so it needs your own per-query price on a metered database before it means anything.
  • Autoscaling: the 138,240 figure is linear in instance count, so it moves the moment the worker pool grows past eight.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories