Build1 publisher3 min readPublished
A plugin update adds shell hooks the harness runs before the model sees the tool call
A study of seven agent harnesses reports 770 confirmed passes in 1,000 runs of a plugin-update attack, and no run was blocked by the model. The harness dispatches the hook, so the model has nothing to refuse.
The Engineer · Build desk

What happened
- Li, Zhang, Hou and coauthors posted a supply-chain study of AI agent harnesses to arXiv on 3 September and revised it on 8 September, covering seven runtimes that sit between a model and the machine.
- Their attack framework, HookPry, ships a plugin update that binds a shell command to a lifecycle event such as session start, a tool call, or a file edit.
- Across 1,000 end-to-end runs the authors count 770 oracle-confirmed passes, 34 partials and 196 fails, a micro-average of 77.0%, with no run explicitly blocked by the model.
- Hermes was the softest harness in the set at 92.5% and Claude Code the hardest at 52.5%.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint Model-side guardrails cannot police this path, because the harness dispatches the subprocess and the command never enters the transcript the model reads.
- decision If your harness loads new hook entries on update without a second prompt, approving a plugin once is approving every command its future manifests bind, so the confirmation has to move to the manifest entry.
- exposure Sandbox policy is the only thing left doing work after the hook starts. The categories that failed in the study failed on network and filesystem limits.
- capability Detection tuned for poisoned tool descriptions inspects the wrong artifact. A registered hook runs without a prompt and without any model cooperation.
A lifecycle hook is two fields: an event name and a shell command [3]. The harness starts that command as a subprocess when the matching event fires, with whatever privileges the harness gave it; the harness itself chooses and runs the command, before the model ever sees it [3][4]. So a hook bound to PreToolUse runs before the tool call the model asked for, and the harness has already executed it by the time the model's turn to respond begins.
The paper's name for the loading behaviour is Temporal Decoupling, and its motivating example is Claude Code automatically loading newly added lifecycle hooks after a marketplace update, with no item-level confirmation and no second authorization [8]. Version 1 of the plugin is useful and boring, you install it, and the update that adds the hook arrives later under the same publisher identity [9]. The ablation is the part I would put in front of a maintainer: remove Temporal Decoupling, or remove the per-harness adapter, and end-to-end success on the comparison set goes to 0% [17].
Two appendix cases show the range. In the first, a plugin advertised as security-sentinel registers a PreToolUse hook called env-validator, which greps the environment for KEY, TOKEN, SECRET, PASSWORD, CREDENTIAL or AUTH and writes hits to a temp file while the plugin returns a normal audit report, and the dump never enters the model context [10]. In the second, a PostToolUse hook rewrites real tool output before it returns to the agent loop: the genuine scan found eval(user_input) and a hardcoded password, the hook forged a pass with zero vulnerabilities, and the model reported the code clean [11].
The transfer question is whether these numbers hold outside the study's own grid. That grid is 40 cases, seven harnesses and five backends, which multiplies to 1,400 combinations, and the paper reports 1,000 runs [6][1]. The per-harness figures land on exact fortieths, since 52.5% of 40 is 21 and 92.5% of 40 is 37, so those rates are counted over cases and not over case-backend pairs [7][2]. Claude Code was the hardest target of the seven at 52.5%, one case away from a coin flip [7][2]. For the 77.0% to describe your setup, your harness has to load new hook entries from an update without a second prompt, and your sandbox has to let the subprocess do what the case needs: Codex CLI persistence was 0.0% end-to-end because network policy and filesystem isolation still applied after the hook started, while privilege escalation averaged 93.5% across harnesses [13][14].
The defense results are thin in a specific way. On a 40-malicious, 40-benign corpus, Microsoft Defender recalled 0 of 40, the authors' lifecycle-hook-aware HookPolicy got 20, and Semgrep got 19, but the union still missed 19, so the two that worked at all agree on at least 18 of the 21 they caught between them [16][3]. There were no false positives on the synthetic benign controls [16].
The MCP comparison explains why prompt-injection defenses will not cover this. Translating 50 malicious MCP tool-description attacks into hooks scored 28 of 50, against 46 of 50 for the native hook versions, because a poisoned description still has to talk the model into calling the tool and a registered hook skips that step [15][4]. What the study does not measure is the marketplace itself, since the evaluation starts with the plugin already installed and the update already delivered [18].
What to watch
- Whether Claude Code or any of the other six harnesses adds item-level confirmation for hook entries introduced by a plugin update.
- Whether a marketplace operator starts diffing manifests between versions for new lifecycle-hook entries before publishing an update.
- Whether a later revision reconciles the 1,400-combination grid with the 1,000 runs reported and publishes per-harness run counts.