The result carrying the most weight in OpenAI's update did not come from a leaderboard. In expert-led assessment against a hardened browser, the company says Astra found previously unknown vulnerabilities and built a full compromise chain that escaped the sandbox and executed commands on the host once the browser opened an HTML file [13]. Against a hardened operating system it found multiple flaws and combined them into a local privilege escalation from an unprivileged user to root [14]. Those are existence proofs, and existence proofs are the right instrument for a capability threshold. A benchmark average would not have been.
The benchmark numbers are weaker than they first appear, and OpenAI comes close to saying so. Contamination was its stated reason for building a fresh internal set [6], which is the correct instinct, because a perfect 100% on the public version [5] leaves no headroom at all and the public test can therefore no longer separate Astra from whatever ships next [18]. The replacement holds 20 items, so a single task is worth five percentage points of measured rate [19]. The comparison against GPT-5.6 Sol is reported in words rather than figures: much higher arbitrary code-execution rates, far fewer output tokens, with neither the rates nor the token counts given [7][8].
Token efficiency is the number I would want most, because it is the one that lands on an attacker's invoice rather than an attacker's CV [16]. V8 is also a peculiar proving ground. It is genuinely hardened and heavily fuzzed, which makes success there meaningful, and it carries an unusually deep public literature of exploit write-ups, which leaves generalisation to unfuzzed internal software an open question. The thing this evaluation does not tell you is whether the same chains fall out of codebases with no public exploit corpus behind them.
The safety case is thinner than the capability case, by its nature. OpenAI delayed parts of Astra's development and release for several weeks to harden protections against both misuse and unauthorized model actions [11][20], and it says retrospective testing indicates the safeguards already in production would have prevented the Hugging Face incident, which Astra was not involved in [10]. A retrospective counterfactual, run by the party with an interest in the answer, is the softest evidence in the document even if it is accurate.
There is also daylight between the criteria as written and the evidence as published. The framework's first condition describes functional zero-day exploits of all severity levels across many hardened real-world critical systems without human intervention [3]; the second describes end-to-end novel attack strategies devised and executed from a high-level goal alone [4]. What has been disclosed is one browser, one operating system and one JavaScript engine, with automated benchmarks and experts in the loop [17]. Either the designation is deliberately precautionary against its own wording, or the fuller record is sitting in the system card promised for launch [15]. I would rather a lab over-call a threshold than under-call it, and precaution costs little at this stage. What I cannot do from this document is size the capability, and sizing is what a defender writing next year's budget actually needs.
