Skip to content

Build1 publisher3 min readPublished

A unique key written before the queue ack stops LLM retries from reordering moderation review

LLM moderation queues need at most one current classification per accepted report, stored under a unique key before the queue ack, a dev.to guide argues. It plans for timeouts, lost leases and racing workers, and keeps unclassified reports visibly pending so a replay can still find them.

The Engineer · Build desk

Illustration accompanying A unique key written before the queue ack stops LLM retries from reordering moderation review

What happened

  • A dev.to guide to LLM triage for a game moderation queue warns that a late or duplicated classification can reorder human review.
  • Its first release returns a compact summary, a policy category and a confidence signal for human reviewers, who stay the final moderators.
  • Timeouts after the provider has finished, lost worker leases, reworded retries and consumers racing on review priority are the failures it plans for.
  • Each result is persisted under a unique constraint on a stable operation key before the queue message is acknowledged.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint At-least-once queue delivery cannot enforce 'at most one' by itself, so the guarantee moves into the database's unique index and the write has to land before the ack.
  • decision Teams have to decide what a moderator sees when the model misses its latency budget, and this design forbids defaulting a timed-out report to safe.
  • cost Reclassification and policy migrations need a second, batch code path with its own reconciliation step, run apart from the live worker pool.
  • exposure Skipping the quality gate leaves the queue open to ranking a credible threat below routine spam, the one error the guide refuses to trade for speed.

Trace each failure against the write-before-ack ordering. When the client times out after the provider has finished, the worker retries and may get different wording back [2]. If the first answer was already written, the retry's insert hits the unique constraint on the operation key, and the existing row stays current [5]. A worker that loses its lease after the answer arrives hands the message to another consumer. The same constraint decides which of the two results counts. A worker that dies before the write never acknowledges, so the message is redelivered and the report is classified on the next attempt.

The post's author, who wrote of being paged by missed jobs and duplicate deliveries in cron and queue infrastructure [11], places the fault outside the model: "The model did what it was asked. The surrounding job failed to preserve a single durable outcome." [3]

The unique key enforces only the "at most one" half. The other half is discoverability. For an interactive report, the guide sets a latency budget matched to the review UI. When it runs out, the report stays visibly pending and is not counted as safe [6]. I'd keep that pending state in the same database as the classification rows. Replay is then a query for accepted reports with no current record. That query still works after the queue has given up on a message.

The word "current" matters as well. It allows reclassification, and the guide sends nightly reclassification, backfills and policy migrations through batch submission [7][12]. The post does not say how the operation key is built. Keyed on report ID alone, a migration's new verdicts collide with the live ones and get dropped. Keyed on report and policy version, the old row has to be marked superseded in the same transaction as the insert, or two rows both claim to be current.

The batch split also protects reviewers. Live arrivals run on a bounded worker pool so a large historical run cannot delay the reports reviewers are waiting on [12], and a batch gives operations one unit to observe and reconcile [7]. Inputs get a boundary too: count or estimate tokens first, split oversized histories at message boundaries, and read context limits from the provider's current model catalog [10]. The guide warns against copying those limits from a blog post, itself included.

The model contract is small. The Go example calls an OpenAI-compatible chat surface, requests JSON, honors Retry-After on a 429 and validates the result before the worker sees it [4]. The struct it validates holds a summary, a category and a confidence score [14]. The guide treats its quality test as a release gate because the costs are uneven: a slower answer can be acceptable, while a credible threat buried under routine spam is not [13]. Vendor choice comes later. For US and EU workloads, the guide says, evidence on data processing terms, retention and regions for the exact model can rule out an option before scoring starts [15].

What to watch

  • Publication of the persistence and replay code, including how the operation key is formed and whether it carries a policy version.
  • Results from the guide's release-gate quality test on how often credible threats rank below routine spam.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories