Branch protection is a list of check names, and that list holds the only real authority in the pipeline. A job that is not on it cannot block a merge, whatever it writes in the thread. So the load-bearing lines in the dev.to author's CircleCI config are not about AI at all: `requires: []` on the ai-review job, which lets it run in parallel and block nothing [11], and `--no-fail-on-issues` on the command it runs, which keeps that job's exit code out of the build result [12]. Only `test-and-lint` is registered as a required check [13]. Credit where it is due: the author names the exit code instead of writing a policy document about trust.
A gate does not need to be accurate so much as it needs to fail attributably. A test suite has a pass/fail history, and a linter has a changelog. A coverage threshold is a number you can graph. When any of them regresses, you can name the commit [9]. The same config pins that floor with `npx nyc check-coverage --lines 80` [14]. Eighty is a debatable number, but it is a bisectable one.
Run the same diff through the same model twice and the verdict can differ [7]. There is no regression suite for the reviewer and no versioned behaviour; nobody wrote down what a good verdict looked like last month, either [8]. A reviewer that starts approving what it used to reject therefore surfaces in the incident review rather than in CI [10]. That asymmetry, not the hit rate, is the argument for advisory position. The tool catches missing null checks and SQL built by concatenation instead of parameters, in seconds, on every pull request [5], and the author reports teams getting fewer round trips out of it as a first pass [6].
The proposed remedy is a golden set: five past pull requests that should have been rejected, five that were legitimately fine, diffs and expected verdicts committed to the repo [15], replayed by a scheduled pipeline against whatever model and prompt is current [16]. Treat that as a benchmark table, because it is one. Ten cases [1] means a single flipped verdict moves the score by ten points [2], and by the tool's own nondeterminism [7] a replay can fail with no change to model or prompt at all, so you need several runs per case before drift and sampling noise are separable. For the set to transfer to your repo, your five rejects have to resemble their five, and theirs come from their own incidents, including the empty-array diff that shipped [1].
The exercise pays off before the first replay runs, because assembling it forces someone to write down what a correct verdict is, which is the artefact that was missing in the first place [8].
In my context this is the right trade. The context: I can bisect a linter, and I cannot bisect a prompt. The fifteen minutes the author quotes for splitting the config [17] buys the one place where that difference gets expensive.
