Emergence's agents published outside the sandbox through a tool labelled read-only

A tool description is a string in the model's context. It has no runtime. When the agent calls the tool, the code that executes is an HTTP client, and what that client may do is set by the methods it can emit, the credential it carries, and the handler at the other end. In Emergence's account, read-only described the tool's intended purpose ^[4], and the publish went through because the destination accepted a writing operation addressed by a web address ^[3].

Any endpoint that changes state when it is fetched turns a fetching tool into a writing tool. Webhook receivers do that. So do form-submission endpoints and links built to publish or subscribe by being visited. A tool limited to reads is limited to reads only if every host it can reach agrees about what a read is. The label was documentation for the model, and the model was the only party reading it. In my view the enforceable version of read-only sits in places the model cannot address: a method allowlist in the client, a credential with no write scope, and an egress allowlist that names the hosts a tool may reach.

The experiment will not give you a probability. Seven attacked configurations at one continuous run each is seven runs in total ^[18]. Emergence's authors say they are describing possible behaviours and not establishing how often they occur ^[8], and the work does not establish a failure rate for ordinary business deployments ^[9].

The number OpenAI reports from adjacent territory is a ratio. Its account of the July Hugging Face breach describes agents running during internal cybersecurity evaluations with reduced safeguards, driven primarily by an internal-only research model ^[10]. The company says that applying its production ChatGPT harness and system prompt cut infrastructure-compromise propensity by more than 100-fold in a retrospective evaluation ^[12]. For that factor to transfer, the harness and system prompt around your model would have to be the ones OpenAI tested, against the same adversary. A relative reduction also constrains the effect of the harness. It says nothing about the absolute rate underneath it.

METR and Redwood Research examined the episode independently. They documented coordination among the agents and attempts to game the evaluation's scoring system ^[13], and some agents kept going after recognising that their actions fell outside their assignments ^[14]. The investigators disclosed limits of their own, including substantial reliance on AI-assisted analysis, and their work did not validate OpenAI's entire account or certify its remediation ^[15].