Skip to content

Build1 publisher3 min readPublished

A ten-gate checklist for agent code places nine of its checks at build time or later

A dev.to checklist of ten software delivery gates for AI agents insists every check be automated, fast, blocking and observable. The gate its author calls no replacement for human review cannot meet that bar.

The Engineer · Build desk

Illustration accompanying A ten-gate checklist for agent code places nine of its checks at build time or later

What happened

  • A dev.to post credited to tamizuddin, first published on tamiz.pro, lists ten software delivery gates it says AI agents will skip unless the checks are baked into the pipeline.
  • Its stated central risk is unreviewed code that bypasses every safeguard humans used to enforce manually. Defective output as such is not the risk it names.
  • Every gate on the list has to be automated, fast enough to finish inside CI timeouts, blocking so failures prevent merge or deployment, and observable in monitoring.
  • Test coverage appears as a minimum threshold enforced in CI, with 80% given as the example figure.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • constraint The blocking rule excludes advisory checks. A review tool its own advocate calls no replacement for human review can fail a build on a finding it emits, and can do nothing about the design flaws the post says agents miss.
  • decision Anyone adopting the list has to decide whether the secret scan also runs in CI, because the pre-commit stage alone depends on a local install.
  • cost Parallelising the gates buys wall-clock time and spends runner concurrency, and the performance gate gets less trustworthy when it shares hardware with the scanners.

The one executable thing in the post is that pre-commit block, and it points at a file. The hook is Yelp's detect-secrets, pinned at rev v1.5.0, invoked with the arguments `['--baseline', '.secrets.baseline']` [4]. The post does not say what belongs in that file. If a baseline is the list of findings the scan is told to accept, the gate's exception list lives in the repository, and anything with commit rights to the tree has commit rights to the exceptions.

Siting is the other thing to read off the list. Dependency scanners "must run in CI, not as a manual afterthought", according to the post [5]; type checking is enforced in CI [6]; Semgrep, Bandit and OWASP ZAP run against every build [10]; Open Policy Agent or HashiCorp Sentinel enforce policy at build and deploy time [11]; `terraform validate` and `helm lint` run at deployment [13]. By my count nine of the ten gates sit at build time or later, and secret scanning is the exception, placed before the commit [17]. The author also requires that every gate be blocking, so that failures prevent merge or deployment [14]. A pre-commit hook fires on the machines where someone installed it. To meet that criterion, the secret scan has to run again in CI.

Eight of the ten gates arrive with at least one named tool [18]. The other two are coverage, specified as a minimum threshold with 80% as the example [7], and performance, specified as benchmarks in CI plus load testing in the pipeline [9]. Those are the two gates where the adopting team writes the check itself. A repo-wide 80% line minimum transfers only if the untested fifth is code you can afford to leave untested, and an aggregate sitting comfortably above the line will absorb an untested pull request without dropping below it.

Blocking is where the list argues with itself. The post offers CodeGuru, SonarQube and GitHub CodeQL as a secondary layer of scrutiny and says they are "Not a replacement for human review" [12]. Such a tool can fail a build on a finding it emits, but not on the subtle design flaw the post says these tools do not catch [12]. The performance gate has a different failure mode: a benchmark can block only when run-to-run variance on the runner is smaller than the regression you want to catch. The post's answer to CI time is to run security scans, type checks and tests concurrently [15]. Concurrent jobs on shared hardware make timing measurements noisier.

On incentives, tamizuddin wrote: "The goal is to make gates fast enough that bypassing them feels slower than complying." [15] That is the right thing for a pipeline to aim at, and it is a design constraint on every check above: a gate that takes twenty minutes will be argued with. The post reports no incident data [19]. Its explanation for leaked credentials is that the model was trained on public repositories that contain them [16]. A reader cannot check that claim from the outside.

What to watch

  • Whether the rest of the post's Q&A, which cuts off mid-sentence on enforcing gates without frustrating developers, addresses who may edit the secrets baseline.
  • Any published rate for how often agent-authored pull requests trip these gates. A number there would move the case from described behaviour to measurement.
  • Whether tooling starts treating .secrets.baseline as a reviewed artifact with its own owners.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories