Build1 publisher3 min readPublished
A single-file acceptance harness reduces agent stdout and stderr to two hashes and byte counts
A post on dev.to publishes accept_run.py, a gate that starts the test command itself and writes the command, directory, exit code and output hashes to JSON, then refuses any repo whose git tree is dirty.
The Engineer · Build desk
What happened
- A post on dev.to argues that an agent's log line is a claim, and lists four fields a merge gate must record: the invoked argv, the working directory, the numeric exit code, and hashes of stdout and stderr.
- It publishes accept_run.py, a single Python file to be saved at the repo root and made the only gate after an agent session, which starts the command itself and writes a JSON report.
- The harness rejects a run on four conditions: a non-zero exit code, an empty git HEAD, a non-empty git status --porcelain, or stdout and stderr both coming back as zero bytes.
- Its behaviour comes from three environment variables, with ACCEPT_CMD defaulting to python -m pytest -q, ACCEPT_ROOT to the current directory and ACCEPT_REPORT to accept_report.json.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Teams adopting the gate have to end the agent session with a commit, because the dirty-tree rule rejects any repo with uncommitted edits before the report can be accepted.
- contradiction The standard the post argues for is stricter than the gate it ships: it calls a zero exit ambiguous, then accepts on a zero exit plus three git and stdio checks.
- cost Auditability now depends on storage someone has to pay for, since the hashes mean nothing to a reviewer unless the pipeline keeps the stdout they cover.
- capability A CI job can branch on the harness returning 1 and compare structured reports between runs, without any step that reads the agent's prose.
The four fields read like a schema, and the implementation fills three of them cleanly. `run()` builds its record from `subprocess.run` with `shell=True`, `text=True` and `capture_output=True` [9], and returns cmd, cwd, exit_code, stdout_sha256, stderr_sha256, stdout_bytes and stderr_bytes [10]. The first field the post asks for is "the argv you actually invoked" [2]. What the report stores is the command string, which a shell then expands [21]. Pass a list instead and drop `shell=True`, and the record holds the vector that was executed.
Capturing both streams also changes what a developer sees during the run. `capture_output=True` pipes stdout and stderr and hands them back only after the process exits [22]. A twenty-minute pytest run prints nothing until it is done, and the output sits in the harness process until then.
The record is hashes and byte counts, not text [10]. A sha256 is checkable only against a copy of the bytes, and the report does not include one [19]. If the CI job archives stdout as an artifact, the hash ties the report to that artifact. If nothing archives it, the hash describes bytes that only the harness ever held.
`ACCEPT_CMD` is an environment variable whose default is `python -m pytest -q` [8]. The post's own warning about exit codes is that a formatter returns 0, a skipped pytest session returns 0, and a `--dry-run` returns 0 [4], so it proposes a sharper test: "Did this exact argv fail?" [5]. The harness can answer that one, because the command string and the exit code land in the same JSON object [10]. The report is JSON, so nobody has to grep it for the word passed [3].
`tree_votes()` records HEAD, `git status --porcelain`, `git diff --stat`, and the sha256 of `git diff HEAD` [11]. `decide()` fails any run whose porcelain status is non-empty [12]. An accepted run therefore has nothing uncommitted, `git diff HEAD` comes back empty, and diff_sha256 is the hash of the empty string on every accepted report [18]. The head and diff_stat fields still vary between runs.
There are no measured numbers here to argue with. The author wrote: "I am not attaching fake timings or fake pass rates." [14] The limit on the evidence is stated in the post itself: "a green run is one sample" [16]. It asks whether CI will share PATH, lockfiles and OS packages with the scratch host where the command passed [15]. For a green run on a laptop to transfer, those three have to come from files the CI job reads rather than from a machine someone set up by hand.
The framing is the part I would keep. The author wrote: "stdout is untrusted speech. Speech is not a run." [1] The harness verifies that a process ran under a recorded command in a recorded directory. Whether that command exercised the change is a separate question, and the post answers it with the tree: `git status` is a vote, `git diff --stat` is a vote, and the chat transcript is not [6].
What to watch
- Whether a later version passes a list argv and drops shell=True, so the recorded command matches what was executed.
- Whether the gate adds a check that the configured ACCEPT_CMD is the project's test command, since a zero exit alone admits dry runs.
- Whether pipelines adopting it archive the stdout the report hashes, without which the two hashes cannot be checked by a reviewer.