Build1 publisher3 min readPublished
Counting silent automation failures requires a run_id that survives the platform unchanged
A dev.to spec defines a silent failure rate that platform run history cannot produce, computed by reconciling events a controller you own fires against receipts posted back by every action in the workflow. It is free under CC BY 4.0.
The Engineer · Build desk

What happened
- A dev.to post describes a failure class in event-driven systems where the platform accepts a unit of work and answers success, the operation never happens, and no run is marked failed and no alert fires.
- One reason is structural: if a trigger stops firing, an event is deduplicated away, or a quota edge swallows it before a run exists, no row is written, and reading rows cannot find a missing one.
- The other is by design: a destination that returns 200 OK with an error object in the body has succeeded at the only layer the platform can evaluate, so the step is recorded green.
- After months of measuring how often this happens on no-code automation platforms, the author published the counting method as a specification covering the metric, its exclusions, and the data contract.
- The rate is missed plus partial over runs accepted and expected to produce output, with events the platform refused at send excluded from both halves and reported separately.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint The measured system cannot be the measuring system, so a reliability figure taken from platform run history is a statement about the transport path and not about whether the operation occurred.
- cost The bill for adopting this is integration work in three places you own, and the per-action receipt edit grows with every step in every workflow you want covered.
- capability Matching each event by identifier catches the case totals-based monitoring hides, where the counts agree and the missing records are the ones that mattered.
Three records and a reconciliation, all written by code you own. A controller writes the event before it sends anything: `run_id`, the workload, which platform copy this event went to, a sequence number, and `fired_at` [7]. Every action inside the workflow posts a receipt back to a destination you control, echoing the event and naming the step that produced it [10]. The destination stamps `received_at` on arrival [11]. The controller also records, per event, whether the platform accepted the send, and if not, why not [12].
That third record is the one an in-house harness usually skips. Without it, an event with no receipt is ambiguous between the platform refusing it loudly, in which case a retry was available, and the platform accepting it and losing it [13]. The author describes a soak run where four sends failed, the harness recorded `send_ok = 0` and threw the error away, and connection reset, timeout and non-2xx are now permanently indistinguishable for those four events [14]. "The fix took ten minutes. The data is unrecoverable," the author wrote [15].
Reconciliation is per event: for each fired event, count the receipts carrying its `run_id` and compare that against what the workload expected [16]. This works only if `run_id` is unique per event and reaches the destination unchanged [8]. A platform that rewrites or drops fields it does not recognise on the way through is not measurable this way [22].
The rate uses two of seven counts, and the spec asks you to publish all seven, because the remaining five are how a reader checks that the two mean what you say they mean [19]. A `filtered` count that comes in suspiciously high, or a nonzero `filter leak`, says the workload is not doing what you think and the rate is measuring something else [19].
Latency here is `received_at - fired_at`. It includes platform processing and both network legs, which makes it an upper bound on the platform's own execution time [11][23]. The author says comparing the two figures is meaningless [11].
A rate produced this way describes one workload on one platform copy, since the event record names both [24]. For someone else's published figure to tell you anything about your automation, the trigger behaviour and the destination's response semantics would have to resemble the ones measured, because those are what decide whether a run gets created at all and whether success can be evaluated [26]. The author says the spec sets out the metric, its exclusions, the data contract that makes it computable, and the conditions a published number has to meet [5], and that the repeated question about the measurements was not about the results but "how are you counting that" [25]. The post is the engineering half [20].
What to watch
- Whether the measurement half of the work gets published with per-platform rates and all seven counts, so the method can be checked against results.
- Whether any no-code platform lets an operator declare what failure looks like inside a response body, which the post says a platform cannot know unaided.
- Whether platforms pass arbitrary fields such as run_id through untouched; those that rewrite them are outside the scope of per-event reconciliation.