Build1 distinct publisher3 min readPublished
The resume flag, the per-item state log and the lock file all shipped in the first commit on 16 March 2026, and the pipeline still could not tell finished work from missing work until an audit three months later.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
Start with what a killed run leaves behind. Every entity it had in flight stays frozen in `running`, and nothing in the system will move it out on its own, so a resumed run sweeps stale `running` back to `pending` before it schedules anything [9]. The reclaim stops at entity state. The lock file the dead run also left has a different owner, and the author says they found that out the hard way [10].
That split is the design in miniature. Five states on one axis [6]; on the other, six failure categories: `timeout`, `quota`, `validation`, `phase_error`, `planning_error`, `unknown` [7]. Keeping the category off the state axis is what makes `paused` and `failed` different verbs, because quota exhaustion is a metered API saying come back later, while a validation error is a deterministic verdict that waiting will not improve, so it stops the run for a human [8].
The hole with the widest blast radius is in the fan-out. The generation step launches many agents in parallel for one topic and waits for all of them, collecting outcomes rather than letting the first failure escape, which the author defends on the grounds that one dying agent should not orphan its seventeen-odd siblings mid-flight [13]. But the runners returned an empty result on permanent quota exhaustion instead of raising [12]. An empty result is a value, and values pass through a gather-all aggregator as data, not as alarms [18]. The batch reported success, missing content was recorded as produced, and the run carried on [12]. The state log simply recorded what the runner told it, and the runner had gotten it wrong.
Separately, three places in the stack each recognised rate-limit errors with their own string matching, so which layer noticed depended on where the error happened to surface; the fix deleted two of the matchers and pointed everything at the third [14]. Three matchers agreeing was luck tied to a maintenance schedule, not something you could test for.
The build order explains more than the bug list does. By the author's own count the circuit breaker after three consecutive quota failures, and the shutdown path that kills child processes rather than orphaning them, predate the runner by eleven days [3], which puts them at 5 March 2026 [1]. The retry that continued a half-written file instead of regenerating it landed four days after the runner [4], on 20 March [2]. That is the confusion in one commit: retry is about surviving a call that failed, resume is about not paying again for a call that already succeeded, and the author admits he assumed for a while that the second came free with the first [5]. Resume here means knowing what is finished across separate runs and across two different machines, which is not the same problem as an agent remembering a conversation across turns [19].
Four bugs in one audit is a claim about this pipeline's shape [11]. For it to transfer you need a parallel step whose outcomes are collected instead of raised [13], a provider that signals exhaustion in-band as a value rather than an exception [12], progress you keep in a state log you parse yourself [1], and more than one machine allowed to start a run [19]. Drop any one of those and the failure set changes into something else entirely.
Ranked by verification strength, evidence, and original report placement.
On 16 March 2026 the batch runner for the author's content pipeline landed in a single commit that already contained a resume flag, a state log with per-item status, error classification, a quota pause, and a lock file.
The author writes that resumability was not bolted on after it hurt: it was there on day one, and it still did not work for another three months.
Eleven days before the runner existed, the project already had a circuit breaker that stops after three consecutive quota failures and a shutdown path that kills child processes when the parent is interrupted instead of orphaning them.
Four days after the runner shipped, the team shipped a retry that continued a half-written file instead of regenerating it.
The author frames retry patterns as being about surviving a call that failed and resume as being about not paying again for a call that already succeeded, and says that for a while he assumed the second came free with the first.
An entity in the system, defined as one topic that owns a set of articles, is in exactly one of five states.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · September 1, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
117 identical errors, zero bugs: when the defect lives in the orchestration1 distinct publisher
build
Three manual interventions in a month, and every guard was working as designed1 distinct publisher
build
Six MariaDB versions, one real difference: the only reason to leave 10.6 is the July 2026 clock1 distinct publisher
build
Force the tool call, then hand Lightsail a long-lived key1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Specific, dated, and entirely self-reported
Every date, state name and bug in this account comes from the engineer who wrote the code, published on dev.to with no commit hash, repository link or second witness. What lifts it above anecdote is how checkable it would be if anything were linked: eleven days before, four days after, six named failure categories, before-and-after pseudocode for the quota path. The weakest link is that the audit — the event the whole piece turns on — is never described at all.
One pipeline, one author, two machines
The design is running, which is more than most write-ups can say, but its entire installed base is the author's own content pipeline: one runner, a handful of topics, sequential runs across two machines that agree only if each pulls committed output first. Nothing in this reporting suggests anyone else has picked up the pattern, and there is no package, repository or third party named.
Sells itself short on purpose
Understated rather than overstated. The build order is offered up as 'not flattering', the file lock is explicitly refused the label of distributed coordination, and the author volunteers that two simultaneously started runs will duplicate work with nothing to stop them. A piece with four silent-failure bugs and three months of broken resume could easily have been staged as a war story; instead the strongest general claim in it is a one-line maxim about rules drifting between implementations.
Reputation is the only stake
No vendor, no funding round, no tool being sold, no sponsor visible anywhere in this write-up — the pipeline it describes has no name and no public artefact to drive traffic to. What remains is the ordinary incentive of a developer-platform byline: engineering credibility, earned here partly by confessing mistakes, which is a mild pull toward tidier narrative shape than a real debugging week usually has.
Believable, unverified
Internally coherent, technically plausible in every particular, and unchecked by anyone. The concrete claims about how the code behaved — empty result flattened into a value, three rival matchers, corrupt state read as zero progress, a check-then-write lock window — are the kind of thing engineers do not invent. Confidence stalls where the story becomes a timeline and a count: three months, four bugs, one day of fixes, all resting on one person's telling.