Build1 publisher3 min readPublished
FFA-001 reproduces a gate that accepts a passing receipt for the wrong commit
GitHub selects required status checks by name alone, so a green badge says nothing about which commit, which workflow, or which changed files it covered. A synthetic fixture shows a downstream gate merging on exactly that gap.
The Engineer · Build desk
What happened
- GitHub's documentation states that required status checks are selected by name and do not take the workflow, matrix leg, or event type into account, which the post treats as the boundary of the control.
- A synthetic pattern called FFA-001, added to Fleet Failure Atlas, models a gate that accepts a receipt by status alone: reproduce mode reports the gate accepting receipt_sha 1111...1111 against candidate_sha 2222...2222.
- Detect mode returns two separate findings, head_sha_mismatch and coverage_not_bound_to_head, splitting the subject question from the scope question.
- Regress mode shows the repaired gate rejecting the stale receipt while still accepting a fresh one, so the fix is not simply a gate that refuses everything.
- The same model is implemented in CI Evidence Gate, a read-only GitHub Action whose local demo builds disposable Git repositories and returns three verdicts.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Because selection is by name, the only thing keeping two same-named jobs in different workflows apart is GitHub's own advice to keep job names unique, which puts a naming convention in the load path of merge policy.
- decision Adopting the fail-closed default means someone owns a changed-path-to-check manifest forever, and every new directory becomes a blocking finding until it is mapped.
- exposure Any repository where a pull request can edit the gate config that judges it has a self-approving path, and closing it costs branch-level protection of the manifest, the verifier, and the invoking workflow.
- capability The failure can now be demonstrated to a skeptical reviewer with no credentials and no production repository, which changes what a design argument about CI evidence has to rest on.
A check run is attached to a commit SHA, and branch protection can require that the check come from a chosen GitHub App [2]. Both bindings are real, but neither covers the producer, which is why GitHub also advises unique job names across workflows: duplicate names can create ambiguous required-check results [3]. Uniqueness as house convention is what the gate is actually trusting.
The post lists four things a display name leaves open: which candidate produced the result, which workflow and concrete job produced it, whether the executed checks covered every changed file, and which trusted policy decided the evidence was enough [10]. Answering the first one means the evidence has to name the exact pull-request head SHA, because a receipt for yesterday's commit is irrelevant even when every test in it passed [11]. Answering the second means recording workflow path, event type, concrete job, workflow run, and latest attempt, so an identically named job in another workflow cannot be mistaken for the required producer [12].
Coverage is a separate axis, and this is the part most CI configs never express. The post's policy sketch maps a surface called python to the patterns `src/**/*.py`, `tests/**/*.py`, and `pyproject.toml`, and requires the check named `test` [13]. The default it recommends is fail-closed: a changed file with no mapping is a finding, not a silent exemption [14]. Three patterns in that block means any changed path outside those three is a finding, so the manifest becomes something you have to maintain deliberately rather than a decoration [20].
Then there is the judge itself. If a pull request can edit the policy that judges it, the verdict is circular, and the fix given is to load the manifest from the base commit, protect the manifest and verifier paths, and keep the workflow that invokes the judge under independent control [15]. Base-held policy answers which policy applies; a protected required workflow or independent review answers who ensures the judge runs at all [16]. Reruns get their own rule: evaluate the newest unambiguous concrete attempt, validate timestamps, and treat retrieval ambiguity as an invalid evaluation rather than a guess, so an older success cannot hide a newer failure or an in-progress rerun [17].
One detail in the fixture is a trap in its own right. In reproduce mode the command exits successfully because it proved its expected condition, so "pass" there means the contained failure was reproduced, not that the vulnerable predicate is safe [8].
What the fixture does not establish is worth stating plainly. It uses synthetic 40-character identities, it is not evidence that GitHub attached a check run to the wrong commit, and it is not presented as a real incident [9]. It also runs with no credentials and no access to a real repository [21]. So the transfer condition is the author's own: much of this is ignorable in a small repository with one obvious workflow, and it starts to bite once there are several workflows, path-specific checks, reruns, generated code, or automation that copies CI results into another gate [19]. The head-SHA comparison and the surfaces manifest are cheap enough that you do not need a base rate to justify them.
What to watch
- Whether GitHub extends required status checks beyond name selection to include workflow path or event type.
- Whether CI Evidence Gate's three verdicts get exercised against a live multi-workflow repository rather than disposable fixtures.
- Whether the fail-closed default survives contact with generated files, which change paths nobody wrote a mapping for.