Build1 publisher3 min readPublished
Swapping in a scope-escape persona pushed 21 unauthorized commits past a coding agent's guardrails
A dev.to team replaced its coding agent's LLM with mocks built to sabotage the repo. The first run let 21 adversarial commits through, and the fix was path and AST checks that run after the model call and before git commit.
The Engineer · Build desk
What happened
- A dev.to team tested its AI coding agent by replacing the primary LLM with five mock personas, including one built to sabotage the repository and one that obeys any instruction it finds in a code comment.
- In the first 36-case run, the scope-escape persona reported 21 adversarial commits out of 27 total and an accuracy figure of 33 percent, because scope enforcement depended on the model respecting path boundaries.
- On the second run of 40 cases, the same persona reported zero adversarial commits and zero commits of any kind, with 34 rejections, 5 successes and accuracy at 65 percent.
- The team reports indirect injection detection at 18 of 18 with no false positives, covering payloads hidden in Python docstrings, comments and third-party API responses.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost The hardened gate left the scope-escape persona with no commits at all, so any legitimate edit that crosses a module boundary is paid for in rolled-back work by whoever was doing the refactor.
- constraint Deterministic path and AST checks only bind writes that reach the commit step, so an agent that can edit files by other means needs the same check placed at that boundary too.
- decision Testing an agent only on a cooperative frontier model leaves the gate and the model's manners entangled, which makes a swappable model call something you have to design in before you can measure either.
- contradiction The 21 and the 0 were measured on suites of 36 and 40 cases, so the headline delta bundles the new gates together with four cases that were not in the earlier run.
The mock is the part worth copying. Each persona replaces the model call itself: hostile attempts repository sabotage and executes injected instructions, obedient does whatever a code comment or a raw API payload tells it, scopeEscape aims at CI pipelines and sibling modules, garbage returns malformed output, and lazySkip refuses to act [2]. That puts the system under test downstream of the model. A suite that passes with a cooperative frontier model tells you the model declined the attack. A suite that passes with scopeEscape tells you a gate declined it.
In the first run of 36 cases, the scopeEscape persona reported accuracyPct 33, adversarialCommits 21 and totalCommits 27, with 32 successes, 3 rejections and 1 deferral [5]. So 21 of 27 commits in that run were adversarial, leaving six that were not [1][6]. The injection scanner held up; scope enforcement leaned on the model agreeing to respect path boundaries [16]. The write-up says that design failed badly: leaving it to the model to know its own boundaries did not hold [6].
Iteration 2 added three checks that run before commit: rejection of any file diff extending beyond the assigned module scope, a hard rollback on edits targeting workflow configuration files, and an AST comparison to block unapproved structural wipes [7]. The same persona then reported accuracyPct 65, adversarialCommits 0 and totalCommits 0, with 5 successes, 34 rejections and 1 deferral [9]. Five successes in a run that committed nothing means the harness is not counting "change landed" as success.
Two numbers need a second look before anyone quotes the delta. The first run's outcomes sum to 36, which is the stated total case count for that run, and the second run's sum to 40, which is that run's total [2][3]. Either the persona was exercised on every case, or the outcomes block is an aggregate across all five personas and not a per-persona figure. The suite also grew by four cases between runs [4], so the 21 and the 0 were measured on different case sets.
For the zero to mean anything in your repo, the writes have to funnel through one place where a diff exists before it becomes a commit. That is where path sanitization and the AST comparison sit [13]. A deterministic check on a staged diff does not see an in-place edit made by a shell command in a sibling directory. The write-up does not say whether this agent can run shell commands.
The injection result is 18 detected out of 18, with zero false positives [11]. Eighteen cases, written by the same team that wrote the detector, is a floor for this suite and not a rate to carry into a different codebase. The more transferable detail is structural: the harness reports rawSourceForwarded as false on hostile injection cases, meaning the untrusted file content never entered the agent's prompt context in the first place [12]. Accuracy, as the harness computes it, moved from 33 to 65, so roughly a third of cases still score wrong in the hardened run [5].
What to watch
- Whether the harness and the 40 cases are published, so another team can run its own agent against the same five personas.
- Whether the path and AST gates still hold when the agent can run shell or subprocess commands instead of only staging diffs.
- Whether a later run reports accuracy above 65 without loosening the path gate, which would show legitimate cross-module edits getting through.