Skip to content

Build1 publisher3 min readPublished

A release-gate scanner printed PASS for two months while scanning zero files

One developer's secrets scanner printed PASS before every release for two months while matching zero files, according to a dev.to post. The fix gives a scan of nothing its own failing exit code, so a broken check stops looking clean.

The Engineer · Build desk

Illustration accompanying A release-gate scanner printed PASS for two months while scanning zero files

What happened

  • The release script had been calling it that way, so every release for two months was verified by a scanner that looked at no files.
  • Exit code 3 now means zero files were scanned, 2 means a path does not exist, and 1 means a real finding, with the file count printed on success.
  • A separate bug had the same shape: a PowerShell-written JSON file with a UTF-8 BOM failed to parse, the error became None, and a dashboard showed an empty table.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • decision Callers that test only for a non-zero exit cannot tell a leak from a check that failed to run, so alerting has to branch on codes 1, 2 and 3 to reach the right owner.
  • exposure Any gate that loops over a glob and fails only on findings can go green on empty input, and each green run raises trust in a check that covers nothing.
  • constraint A checker's usual tests feed it inputs that match, so they pass with this bug present; only a zero-match test that asserts failure exercises the path.

The scanner had one failing branch, `if findings: sys.exit(1)`. Everything else fell through to `print("PASS")` [1]. Its walk was `glob.glob(root + "/**/*", recursive=True)` [1]. That pattern assumes `root` is a directory. Handed a file path, it becomes `some/file.mp4/**/*`, and the author reports it expands to nothing: zero files scanned, zero findings, PASS, exit 0 [2]. The loop body never ran, so the failing branch had nothing to test.

The count was on screen the whole time. Nothing errored or warned, and the log line printed `files scanned: 0` [4]. It is a rare bug that prints its own diagnosis every time it fires. The author wrote, "I had read past it every single time, because the word next to it was PASS." [5] The post describes the failure as failing open: a guard that breaks stops blocking, and stopping is silent [7]. It also argues that each green run is taken as more evidence the check works [7].

The fix is four lines. If `scanned == 0`, the script prints a BLOCK message and exits 3 [8]. A path that does not exist exits 2 [10]. A real finding keeps exit 1 [9]. I would copy the split before anything else. The author's point is that 1 means "found a problem" and 3 means "could not do my job", and that collapsing both into "non-zero" discards the signal that would have caught this [9]. A release script that only checks for non-zero still blocks on 3. The separate code matters in alerting, where a broken check and a leaked secret belong to different people.

The success path changed too. It now prints `[verify] files scanned: {scanned} | findings: {len(findings)}` [11]. A reassuring result arrives with its denominator. "If it cannot tell you a number, it is not evidence," the author wrote [12].

Adopting the guard has a cost the post does not work through. A hard block on zero matches assumes zero is never a legitimate answer. The published wrapper runs as `python zero_match_guard.py --paths "dist/**/*.js" -- npm run lint` [17]. On a branch that builds no JavaScript, that job now fails. The author applies the rules to anything that gates a release, a deploy or a payment [15], and for those I think a hard block is the right default. The exemption should be an explicit flag in the job definition, where a reviewer can see it. The guard itself is one file using only the standard library [17].

The second bug in the post has the same shape. A JSON file written by PowerShell carried a UTF-8 BOM. `json.load(open(path, encoding="utf-8"))` raised. The exception was caught and turned into `None`, and a dashboard rendered an empty table [13]. The post gives the fix as `encoding="utf-8-sig"` [14]. That handles the BOM. I would also remove the `except` that converted a parse error into `None`, because the next malformed file would produce the same empty table.

The post's checklist ends with one regression test: feed the checker an input that matches nothing, and assert that it fails [15]. The author shipped the zero-file bug three times in the same codebase [6], and wrote that this test is "the only one that would have caught all three of my versions of this bug." [16]

What to watch

  • Whether zero_match_guard.py adds an explicit, per-job override for checks where an empty file set is a legitimate result.
  • Whether the guard's shipped regression test covers the file-path-instead-of-directory input that caused the original two-month miss.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories