A document checker aimed at the wrong path reported zero problems and exit 0

Every incident in the list was a correct program producing a true statement ^[3]. A checker that counts problems inside a set it discovered for itself reports a numerator without its denominator. "scanned 0 documents, 0 problems" was true when the author pointed his document checker at a file where it expected a directory, and the run exited 0 ^[5]. The day before, the same tool had reported green over 133 documents, so he believed it ^[6]. The author's summary of why this class is expensive: "a red check gets fixed, and a green check gets built on" ^[4]. His fix is to fail closed when the denominator is empty, exit 2 with E_NOTHING_SCANNED, and treat "zero items, therefore zero problems" as an error condition and never a pass ^[7]. The shell version is the cheapest to reproduce. `N=$(some_command | wc -l)` prints "0 items" and exits 0 when the command does not exist, because the exit status belongs to `wc` ^[22]. wc counted the lines it was handed, and there were none. He hit this twice in one day, both times on the question "how many items are waiting for me?", and both times the true answer was not zero ^[23]. He caught it only because he had a second, unrelated way to count the same thing ^[23]. The fix is `set -o pipefail` and a die that reports the result as UNKNOWN instead of 0 ^[24]. A `grep -c` for `E_` in an error log counted `A_NO_FENCE_ADVISORY`, because `E_` occurs inside `FENCE_ADVISORY`, and he spent a while convinced the tool was miscounting ^[20]. The rule he took from it is to suspect the ruler before the subject when a measurement surprises you ^[21]. The narrow denominator came back four times in three days, in four disguises ^[12]. Grepping his own file titles for a business goal's name said the work was two days old. The three matching files were internal quality reports that mentioned the goal in the title, and the real answer was thirty-three days ^[13]. A check for accidentally future-dated timestamps returned three hits, all of them policy expiry dates in the content, where the real answer was zero ^[14]. A leak check found no internal identifiers and he called it passed; a reviewer pointed out that he had measured names while the actual risk was method, an axis the checker never looked at ^[15]. None of the four has a code fix, and the rule he wrote instead is to state what you excluded on the same line as the number ^[19]. Checking all ten of the stamped index lines properly turned up five genuinely stale ^[17], so half of a set advertised as verified was out of date ^[28]. The post credits someone else for the phrasing it adopts: "putting today's date on a line you did not actually check is itself the false green" ^[18]. This is one engineer's tally of his own tooling, about seven a day for three days ^[27], and the post says where he did not measure something ^[2]. For the count to say anything about another pipeline, the checks there would need the same three shapes: a count over a set the check discovers itself ^[5], and an exit status routed through the last stage of a pipe ^[22]. The third is a match on a line rather than a parsed value ^[25]. The permission monitors are the clearest instance of the third. One matched a stray quotation mark through a `[^H]` character class, and the other searched for "clear" and "authorize" and hit the line that imposed the restriction, because that line contains both words ^[25]. His replacement contract uses three exit codes, 0 for released, 1 for still held, 3 for could not find the key at all ^[26].