A duplicate process wrote for ten hours because the watchdog compared strings after formatting

The duplicate-process watchdog existed to stop a second copy of the process from starting. It compared strings after formatting instead of before, so the duplicate did not look like a duplicate to it, according to the author of a post on dev.to ^[5]. Both copies ran side by side for ten hours. Both were writing, and neither complained ^[4].

Two of the five failures were failures of attachment. A component was still installed and still configured but no longer attached to the thing it was supposed to be attached to, and nothing logged it ^[7]. A file was written, correct and current, for a consumer that had stopped reading it months earlier ^[8].

The message loop had the same defect in a different place. It checked whether a response contained items, and went back to sleep when it did not ^[18]. A revoked token returns an error. So does rate limiting, and so does a second instance stealing the connection ^[19]. One sleep branch therefore served four conditions: three failures and one genuinely quiet poll ^[24]. "The process stayed alive, the watchdog stayed green, and the system was completely deaf," the author wrote ^[20].

The startup-order case is the one with timestamps. The journal shows the component loaded at :29, the value read at :36, and the connection authorized at :39 ^[13]. The read beat authorization by three seconds ^[14]. It returned a clean, plausible zero, the system used that zero as a measurement, and it configured itself three hours away from correct ^[12].

Two of the five interventions happened only because something restarted ^[10]. The current practice is that every scheduled job writes a heartbeat on every run, including runs where it decided there was nothing to do, and that the absence of a heartbeat for longer than the interval is itself an alert ^[17]. That rule requires the alerting side to know each job's interval.

The path failure is the cheapest one to reproduce. The author moved a directory; the wrapper script coped because it resolved its own location at runtime, and the registered scheduled tasks failed because they held absolute paths recorded at registration time ^[15]. The machine said nothing about it ^[15].

The notification pipeline deleted each event from disk before confirming it had been sent, so a failed send left no event and no retry ^[22]. "I'd built at-most-once delivery by accident, in the one place where at-least-once was the entire point," the author wrote ^[23].

Five interventions in two days is one operator's log, and the post does not say what the system does ^[2][25]. For these failures to be yours, some ordering has to match: your comparison runs after normalization, your scheduler stores absolute paths at registration, your startup reads a remote value before it checks that the connection is usable. The author frames the gap as the one between "automated" and "passive" ^[1], and states the test this way: "if the only evidence of health is the absence of errors, you have no evidence of health" ^[26].