Build1 publisher2 min readPublished
git bisect run narrowed twelve commits to one sign-stripping diff in four test runs
Git's binary search closed a twelve-commit range in four automated test runs. It holds up only when the test exits 0 on pass and still runs when Git checks out a commit from a week ago.
The Engineer · Build desk
What happened
- A test that had passed a week earlier failed on main after twelve commits, most of them documentation, with no one recalling which commit touched the code.
- Told that HEAD was bad and c7c12bd good, Git checked out the midpoint commit and reported five revisions left to test, roughly three steps.
- Handing the script to git bisect run instead of answering good or bad by hand closed the search in four test runs, naming ee25549, "Perf: simplify addition loop", a one-line change to calc.sh.
- That line replaced total + n with total + ${n#-}, which strips a leading minus sign and turned every negative argument into a positive.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint How far back a team can search is bounded by when the test was written, because Git checks out the old commits and a test file added last week is not there.
- exposure A mislabelled good endpoint yields a wrong culprit with exactly the same confidence as a right one, so the two endpoint runs are the cheapest part of the search.
- capability Pointing run at one failing behaviour rather than the full suite makes each probe cost one test, which is what lets the technique work in repositories where the suite takes twenty minutes.
- decision Before wiring a third-party runner into bisect run, someone has to check what it returns on failure, because two codes in the usable range mean something other than bad.
The exit status is the whole interface. If the test is a command that exits 0 on pass and non-zero on fail, as the demo's `test.sh` is, the operator never answers a prompt [19]. `run` reads that status and nothing else: 0 marks the commit good, 1 through 127 mark it bad, 125 means the commit cannot be tested and gets skipped, and any other code aborts the bisect, which the write-up takes from the git manual [12]. Two of those cases are traps when the command you hand to `run` is not one you wrote. A harness that exits 125 on a failing test is recorded as unbuildable. One that exits above 127 stops the search instead of marking the commit bad [4].
Four probes for twelve commits is what the method predicts: ceil(log2 12) is 4 [1]. The count does not depend on where the bad commit sits, and the source puts it at the same three or four steps whether the culprit is the second commit or the eleventh [7]. Twelve diff reads against four test runs is three times fewer inspections [2], and the gap widens with the range, since 200 commits need eight probes [3].
The repository was built for the demonstration, on git 2.43.0: a shell script that sums its arguments, a test asserting `2 3 -> 5` and `10 -4 -> 6`, and twelve commits of which ten only append to `NOTES.md` [3][4]. Only the second assertion involves a negative number [4]. For the four runs to transfer to a working repository, the test has to run at every commit in the range, and `bisect run` checks those commits out, so a script added last week is not present once Git walks back past it. The source's fix is to keep the script in a temp directory and call it by absolute path [13].
Commits that cannot be built come out of the range by hand with `git bisect skip`, and Git works around them [16]. `git bisect log > bisect.txt` writes the session to a file, and `git bisect replay bisect.txt` re-runs it later [17]. Merges are handled as well: bisect walks the DAG, and the first bad commit can be a merge, which the source calls a real answer [18]. `git bisect reset` puts the tree back where it started [8].
What to watch
- A range where the breakage is intermittent or spread across two commits: the four-probe count assumes one first bad commit.
- Ranges containing merges, where bisect can name the merge itself and the diff to read is larger than one line.
- The recovery path when a bisect starts on a dirty tree, which the source's lab covers through the reflog.