Plan mode is a plausible lever for a mechanical reason. Default mode, which Claude labels manual, Cursor labels agent and Codex labels default, ingests the prompt and starts writing straight away [5]. Plan mode inserts a proposal step, and in this experiment the researcher always accepted the recommended or top option when the agent asked [4]. So the treatment under test combines the model thinking first with whichever plan each vendor decided to put at the top of its list, not "the model thinks first" in isolation.
Then the denominator. Three models in two modes is six applications, one build per cell [13]. One build per cell is one draw against one draw, which means a gap between the plan column and the default column is not yet separable from the variation you would get by running either mode twice. The write-up presents itself as the first post in a planned series [1]. The limit here is on what a reader can carry into a configuration decision this week, not a mark against the work itself.
One finding does not need replication to stand, because it repeats. In the Sonnet 5 default build, the review put the insecure direct object reference at the top: the GET /:id and /:id/download routes never verify the requester against the asset owner, while the DELETE path does [10]. Ownership checking was applied to the destructive operation and skipped on the read paths, which looks like a learned convention rather than one careless file.
Two things complicate the per-mode scoreboard that later posts will presumably publish. The audit ran Claude's code_review skill on Sonnet 5 over every iteration [8], and Sonnet 5 is also one of the three models under test [2]. For a flaw common to all six, that matters little. For counting findings mode by mode, a single auditor family is thin. The dependency layer was not left alone either: the researcher reports that the Supply Chain Firewall substitution had a small effect on the security output, and kept it in light of recent supply chain attacks [7]. Composition findings here sit on a floor an unassisted agent would not have had.
The strongest evidence against the mode hypothesis comes from the one arm the material describes in full. Sonnet 5 in default mode produced a double-submit CSRF cookie wired into every state-changing route, four separate throttles for authentication and file actions, and stateless JWTs held apart from revocable refresh tokens in the database, and the researcher calls it the most resiliently designed backend of the whole test group [11]. The fast path, with no planning step, returned the strongest backend of the six.
My read, conditional on the arithmetic above: mode selection is a weak lever next to whether anything in the pipeline checks ownership on read routes. I would revise that if the series reports a consistent per-mode gap over many runs of the same prompt.
