Build1 distinct publisher3 min readPublished
A developer ran the same implementation plan through four reasoning conditions in Codex CLI, kept the high-effort code because it alone fixed a preparation step that invalidated its own result, and still intends to default to medium.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
A green test suite can hide a self-invalidating preparation step, and that is exactly what happened here. Galley can prepare a task's environment and write acceptance-test skeletons tied to the requirements before an implementation agent starts [8]. All four conditions added fingerprints so a retry could tell whether the inputs had changed [9]. Galley also writes into the task while it runs: creating a skeleton appends an explanation to an acceptance criterion, and if that explanation lands in the next fingerprint, the successful preparation step invalidates its own result [10]. Low, medium and Sol each left a version of that in the setup reuse path [11]. High separated the generated explanation from the user-supplied contract and traced the change through preparation and task updates into the next run [12].
Getting that wrong carries a real cost. An unnecessary preparation run buys another wait and another model invocation, and it can send the test creator back into files the implementation agent has already edited [13]. Preserving valid reuse was in the requirement alongside detecting changed inputs, so the three conditions that missed it missed half the task [14].
Seventeen minutes separates the two implementation runs, about 55 percent more wall clock for high [20]. What would have to be true for that number to transfer: your tasks would need the same shape, since the procedure as described ran each condition once, on one repository, against one plan, with no repeated trials reported [21]. The 17 minutes is a single data point, not an average. If your codebase has no retry path and no preparation to reuse, high's extra minutes buy you nothing.
The tiers also do not rank the same way in every phase. Galley deliberately lets an operator requeue a running task; low and medium proposed restricting that operation, while high checked the documented intent and the existing test and left it out of its formal findings [15]. A tier that reads the test before proposing to remove a documented manual override is doing product review, which is not what an effort setting advertises. Then the ordering flips: in a separate review pass over identical copies of Sol's implementation, medium found a startup failure high missed [4].
High's remaining choices were unglamorous in a way that reads well. For large Git change sets it extended the existing mechanism for passing paths through standard input; for corrupt task records it reused the existing isolation logic [17]. More behaviour covered, no new subsystem per case. The author also declined to score findings or changed lines as a proxy for quality, and credited deliberate exclusions only where the report or visible session history backed them [16].
This run shows something narrower than a general reliability gap. On the record in this write-up, high is slower by 17 minutes and missed one startup failure in one review; it also produced the only correct reuse path and the implementation being carried forward [6][12]. That branch is now in a Galley PR whose first CI run passed on Linux and macOS and failed on Windows when a timeout test tried to clean up a log file [18]. The default the author states is medium; the code he is keeping came from high [6].
Ranked by verification strength, evidence, and original report placement.
The four conditions were GPT-6 Astra at low, medium and high reasoning effort plus GPT-5.6 Sol at high. Each first analyzed the repository for bugs, performance problems and maintainability improvements. The author selected a common implementation scope from the Astra findings and gave all four the same plan in separate worktrees. Finally, fresh sessions reviewed identical copies of Sol's implementation.
The sessions ran in Codex CLI. The agents read guidance from codex-workflows, including its coding, testing and development-judgment skills, together with Galley's repository instructions and quality profile. The plan stated the expected behavior and constraints; the agents chose the design, order of work and verification.
High's implementation run took about 48 minutes against medium's 31.
In a separate review of the same Sol implementation, medium found a startup failure that high missed.
High handled the interactions between retries and persisted state more completely.
GPT-6 Astra at high reasoning effort produced the implementation the author decided to keep, and the author still plans to use medium effort by default.
Distinct publishers with included, body-backed reporting in this cluster.
1 article · September 4, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
invest
OpenAI rates GPT-6 Astra capable of hacking hardened systems without human guidance1 distinct publisher
build
Superpowers makes spec-driven work a precondition, then ships it to twelve harnesses1 distinct publisher
build
2,513 tool calls, zero refactorings: what agents actually do when you ask them to refactor1 distinct publisher
science
Two harnesses put the same model 37 points apart on ARC-AGI-32 distinct publishers
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
One practitioner's runs, checkable in patches
Everything traces to one developer describing his own sessions on his own repository, and he does show his work where it can be seen: an open Galley pull request, a named Windows CI failure, per-condition minutes and dollars, and a defect specific enough to argue with. What is absent is any second pass. No condition ran twice, nobody else graded the same four outputs, and the scoring criteria and usage breakdown sit in evaluation notes the post links rather than reproduces.
One developer's default setting
Adoption here amounts to one person changing a configuration flag. The high-effort implementation exists as a pull request its author says needs more review and that already failed CI on Windows, and the change he actually intends to make is to run medium by default. No other team, repository or product appears anywhere in the reporting.
Buries its own best finding
The framing sells a timing gap while the durable result — a cache key that included a value the cached step itself wrote — arrives two-thirds of the way down and is described almost in passing. Pulling the other way, a single run per condition becomes advice in the title, and the author's 'for my workflow' qualifier does not travel with a headline that reads as a general recommendation.
His own tools under his own test
He owns the codebase and the workflow skills the agents draw on; the models themselves are the one outside element. Galley and the codex-workflows guidance the agents read belong to the author, so the comparison doubles as a walk through his own projects, and dev.to publishes it without an editor between him and the reader. Against that, the tier he lands on is the cheaper one at $15.61 rather than $21.03, and he volunteers the case where his preferred setting lost, a detail that cuts against reading this as a promotional write-up.
Specific enough to trust, narrow enough to bound
We read the account as honest because it is falsifiable in the details: named code paths, a reproduction through the real daemon startup function, and a loss recorded against the author's own favoured condition when medium's reviewer caught a startup failure high missed. Our confidence stops at reach. One plan, one repository, one attempt per condition, and the same person grading the code and writing the tool being repaired.