zerocase refuses green CI runs whose reports show nothing executed

A JUnit header such as `<testsuite tests="50" skipped="50">` describes a green run in which nothing executed ^[6]. A stdout pattern counting "50 tests" matches exactly that run ^[6]. The author, who publishes the code as Megapixel99/zerocase, wrote that the report parsers add one distinction: "a total is not a denominator" ^[7][21].

The hard case is a `<testcase>` carrying both `<skipped/>` and `<failure>`. Quarantined flakes, xfails and reruns that gave up all leave that shape, and zerocase counts it as not run ^[8]. According to the post, counting it as both is how a suite that skips everything ends up looking like a suite that broke everything ^[8].

The failed count is printed and never used as a verdict ^[9]. A report in which every test failed still clears the floor, and one test in the suite asserts exactly that ^[9]. I think this is the right split. The runner's exit code already reports pass or fail. It cannot report that nothing ran ^[1][9].

A report can hold a number, hold a zero, or be unreadable, and zerocase keeps the third state separate ^[10]. Score an unparseable file as zero and builds fail for the wrong reason. Score it as a pass and you have the defect the tool exists to catch ^[10]. The freshness rule has the same motive. In the author's words, "yesterday's junit.xml parses beautifully and says four hundred tests passed" ^[22]. On the override flag, the author wrote: "The name is deliberately unpleasant" ^[23].

Header attributes get the same suspicion. A runner killed halfway can leave `tests="47"` from one run above `<testcase>` elements from another. When the two disagree, zerocase counts the elements and prints the discrepancy ^[12].

The test design is good engineering. The committed controls are a pair. A fresh report with tests in it is evidence, and a fresh report with nothing in it is did-not-run ^[13]. Because the two are asserted together, a gate that always passes fails, and so does a gate that always refuses ^[13]. A parity suite sends one fixture table over stdin to both language halves and compares tallies and output sentences character for character ^[14].

The author wrote that the two first-run survivors of the mutation pass were worth more than the 11 catches, which came to about 85% of 13 ^[15][20]. The empty-glob mutation survived because its test reached a verdict through the freshness gate. That gate fires first and returns before the empty-glob refusal runs. The branch was reachable through `--allow-stale` and exercised by nothing ^[16]. The skipped-counted-as-failed mutation survived because no fixture held a testcase with both a skip and a failure ^[17]. A third mutation, on `self.tally = self.tally or {...}`, is a no-op on the first run. It scored SURVIVED and looked like a test gap while proving nothing ^[18]. "A mutation suite can lie in the flattering direction as well as the other one," the author wrote ^[19].

On paper the adoption cost is small. zerocase installs from pip or npm and has one dependency, didrun ^[3][4]. didrun asks the same question of stdout using a regex the user supplies. zerocase reuses its states, exit codes and freshness logic ^[4]. For the gate to work in another pipeline, the runner has to emit one of the five supported formats and write the report during the wrapped run ^[5][11]. The evidence in the post comes from the author's own controls and mutation pass. It does not include results from other teams' pipelines ^[13][15].