Skip to content

Build1 publisher2 min readPublished

Claude runs five of the seven checks on Claude-written code at Anthropic

Boris Cherny named six guardrails for Claude-written production code at Anthropic. Almost everything else in circulation about how they fit together comes from the dev.to post that relayed his quote.

The Engineer · Build desk

Illustration accompanying Claude runs five of the seven checks on Claude-written code at Anthropic

What happened

  • Anthropic's Boris Cherny said production code written by Claude should be held to a higher bar than the same code written by a human.
  • The guardrails he listed are lint rules, Claude-driven end-to-end tests, Claude-powered fuzzers running daily, automated code reviews, automated security reviews and automated code refactoring.
  • The dev.to write-up that relayed the quote adds a standard unit and integration test stage to the pipeline, and says it presumes those tests are human-written or generated alongside the code.
  • That post's example for running two separate review passes is a function that is clean and readable and still vulnerable to SSRF, which it says the security pass is the one to catch.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint A model that writes both the code and the tests can produce a suite that agrees with its own mistakes, so the green result has to be validated by something the model did not write.
  • cost Five of the seven stages are model calls and the fuzzer runs every day, so a team copying the list takes on recurring inference spend per change, and there is no published figure to size it against.
  • decision Anyone adopting this stack has to write their own merge-blocking rules from scratch, because the severity ladder now circulating with it is a blogger's guess.
  • contradiction The post details independent layers and a blocking order, and presents the stages it cannot source as inference, so those mechanics cannot be cited as Anthropic practice.

A fuzzer is worth running when it finds inputs nobody would write by hand. The dev.to post says a Claude-powered fuzzer "likely" does one of two things: read a function signature and generate cases for null inputs, boundary values and type mismatches, or mutate existing inputs with knowledge of their structure, so that a JSON or SQL payload comes out plausible but malformed [5]. Both of those are semantic test generation. The post marks its own description as inference.

It also names the failure mode. If Claude generates test cases that always pass, the fuzzer becomes a false sense of security, and a separate validation layer is needed to confirm it is finding bugs and not green checkmarks [6]. I would build that validation layer before the fuzzer. Running daily means new functions get new cases and refactored code gets re-fuzzed [12], so the suite grows whether or not its finds are real.

The split between the two review passes is the one design choice with a stated reason. Per the post, the threat models differ. A code review asks whether the logic is clear, whether names are descriptive, whether a function does one thing, and whether there are off-by-one or null dereference errors [8]. A security review asks whether an attacker can inject SQL or shell commands, whether authentication checks can be bypassed, whether sensitive data leaks into logs, and whether rate limits are enforced [8]. Two passes let each one carry its own prompt, its own examples and its own escalation rules [15].

The post says Anthropic did not state whether a Claude-flagged security issue blocks the merge, notifies a human, or gets rewritten and re-submitted automatically [7]. The severity ladder it offers is its own guess [7]. The ordering claims sit in the same category: that each layer operates independently, and that the refactoring step does not run until a flagged SQL injection risk is fixed, are the post's construction [4].

Anthropic's own account of this is one sentence and six nouns [1][2]. Five of the seven stages in the post's pipeline are model calls [13]. The list transfers to another team only if their existing tests are good enough that an agent-written end-to-end suite adds cases instead of restating them. The second condition is that the two review prompts differ enough that the security pass catches what the code pass approved. Someone also has to own the question of whether the fuzzer's finds are real. The final stage, in the post's telling, is one Claude instance flattening another's five levels of nested if statements into early returns or guard clauses [11].

What to watch

  • Whether Anthropic publishes its actual escalation rules: which severities block a merge and which only notify a human.
  • Whether anyone reports a bug count from the daily fuzzer, or a share of merges the automated security review stops.
  • Whether the agent-written end-to-end suites get measured against deliberately injected faults, which is the check the post says is needed and does not describe.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories