Build1 publisher2 min readPublished
A delivery row per consumer records which credential read each webhook event
One registration, one signed POST into Postgres, and a pending row for each internal reader. The design costs four row writes per event and makes acknowledgement a fact an auditor can still query a year later.
The Engineer · Build desk
What happened
- The design registers one webhook endpoint, verifies the signature, stores the raw body keyed by the provider's event id, then writes one pending delivery row for each registered internal consumer.
- The receiver does four things and returns 204, with no consumer logic inside the request; on an exception it rolls back and answers 503 so the platform retries on its own schedule.
- Consumers drain their own unacknowledged rows in batches of 20 using for update of d skip locked, so several workers of one consumer can run without both claiming the same event.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint At-least-once delivery means every handler has to be idempotent before this design is safe. The provider event id guards the events table, not the billing call a consumer makes.
- cost Per-consumer attribution is paid in write volume: four rows per new event at three consumers, plus a full copy of the body, and the count rises with every internal reader you add.
- exposure The credential in the audit row is read from the consumer's own environment, so the trail is only as good as the deployment isolating those credentials from each other.
The two lines that carry the ingest path are `on conflict (event_id) do nothing returning id` and the `if (ins.rows.length)` guard beneath it [7][8]. Together they make redelivery cheap: a second POST of the same provider event id inserts nothing and fans out nothing [7]. They also freeze the consumer list at the moment the event first landed. Add a fourth name to `CONSUMERS` tomorrow and yesterday's events have no pending row for it [2].
With the three consumers in the listing, billing-sync, seat-provisioner and usage-ledger, a first-seen event writes four rows inside one transaction: one event plus three deliveries [6][1]. The event row also stores the full raw body as text [18]. Attribution costs one row per consumer per event, so ingest write volume rises with the number of internal readers, not with traffic alone [1].
The author's argument for the outbox table over a broker acknowledgement is about what survives. A broker ack is an opaque counter; a row update can carry the consumer name, the credential that did the work, the attempt count and the timestamp [16]. "A row update is a fact you can query a year later," the author wrote [15].
Everything here assumes at-least-once delivery, which the author treats as what every real platform and every real queue actually gives you [14]. Consumers must be idempotent. "There's no negotiating with that," the author wrote [14]. The provider event id keeps a duplicate insert out of the events table [7]. Guarding against a handler that charges a card twice is the handler's own job.
The credential in the audit row comes from `process.env.CONSUMER_PRINCIPAL`, read by the consumer process and described in the listing as that consumer's own credential [13]. Only the consumer process ever reads it. So the row records the name a process was configured with, and for the auditor's question to be answered by that row, each consumer has to run under a distinct credential that cannot write another consumer's deliveries [3].
The concurrency choice is sound: `for update of d skip locked` lets several workers of the same consumer drain in parallel without two of them claiming the same event, in batches defaulting to 20 [12]. The signature check compares digest lengths before calling `timingSafeEqual`, so a truncated hex header returns false instead of throwing [11]. Failure in the transaction rolls back and returns 503, with the comment that the platform retries on its own schedule [9].
The published drain listing ends mid-statement, before the acknowledgement update, so the write that sets `acked_at` and records the principal is not shown [19].
What to watch
- A reported write rate from a live deployment would show whether the four-row ingest transaction holds up on an ordinary primary at peak webhook volume.
- A published backfill path for consumers registered after ingest, since delivery rows are written only when the event first arrives.
- A retention policy for the raw payload column, which stores the full body of every event in the same table the auditor queries.