Leadership1 publisher3 min readPublished
Anthropic's most capable model built working exploits from 16 of 39 published patches
Anthropic's red team says the scarce reverse-engineering skill that used to give defenders weeks is no longer the bottleneck. It measured that on Firefox and Windows kernel fixes, where a 19-day median gap counts as fast.
The Board Room · Leadership desk
What happened
- Anthropic's Frontier Red Team measured how far large language models can accelerate and automate the development of exploits for vulnerabilities that have already been disclosed and patched.
- Across 18 recent Firefox security patches, Anthropic's most capable model, Claude Mythos Preview, autonomously built eight working code-execution exploits.
- On 21 Windows kernel patches, the same model produced eight full exploit chains that escalated a low privilege user all the way to full SYSTEM control.
- The median gap for the patches studied was 19 days to release, which Anthropic calls fast by industry standards, where enterprise vulnerabilities typically take many weeks or months to remediate.
- The historical benchmark Anthropic cites is far slower: WannaCry hit 59 days after the MS17-010 patch shipped in 2017.
Compiled by The Board RoomSomething wrong?How this is made
Why it matters
- decision Anthropic's recommendation is to deploy patches faster. The remediation calendar becomes a signed decision about how many days of exploitability the company accepts on each asset class.
- exposure Any estate remediating on a multi-week cycle sits well outside the window Anthropic describes as fast, and the exposure is to already-published bugs rather than to novel research.
- constraint Withholding source code no longer delays exploitation the way it used to, because the Windows kernel chains were built from patches whose source is not available.
- contradiction The same post says the bottleneck has largely fallen away, and that target discovery, delivery and detection evasion still cost attackers time. It supports both a shorter patch SLA and a slower read of end-to-end campaign speed.
Sixteen working exploits across 39 published patches is a 41 percent hit rate, 44 percent on the Firefox set and 38 percent on the Windows kernel set [24][22][23]. Windows kernel fixes ship without published source, and Anthropic reports the model worked from the compiled patch anyway [9].
Anthropic scored the evaluation in successes per patch set, while a remediation window is set in days. It also selected for age: only bugs whose fixes had been public in Mozilla's source repository for at least 90 days were kept, with the exploits verified against jsshell, the JavaScript engine's standalone command-line build, instead of the full browser [18][19]. The 18 Firefox bugs came from the security patches shipped in Firefox 148 and 149, released February 24 and March 24 [17].
Firefox is close to a best case for defenders by the post's own account. It downloads fixes in the background, adopting one takes a browser reboot, Mozilla ships a one-off release when a fix cannot wait for the schedule, and dot releases recently moved from a monthly to a roughly weekly cadence [14][15]. The authors then generalise from it. "If even these patch gaps are wide enough for attackers to exploit, then we can be confident that most other software's gaps are too wide, too," they wrote [27]. Their recommendation: "Defenders should try to accelerate how quickly they deploy patches in response" [12].
The comparison class runs to weeks: the public exploit for Citrix Bleed took about two weeks in 2023, and in Mandiant's 2020 analysis of N-days, 16 of 25 took a month or more to exploit [6][7]. Patch diffing was slow, specialised work, and that slowness gave defenders time to roll their updates out [4]. "With frontier models, this bottleneck has largely fallen away," the authors wrote [11].
Anthropic measured its own models, using a hardened harness and grader it had already built for earlier work with Mozilla, and published the defensive conclusion itself [20]. The same post narrows its own claim. It notes that target discovery, delivering the exploit to the target and detection evasion all take time and resources, and that exploit development was the step most bottlenecked by scarce reverse-engineering expertise [13]. Anthropic also reports that its public models, with safeguards turned off, built exploits too, fewer than the preview model managed [10].
Shortening a remediation window costs soak time, the testing that catches the patch that breaks production. The choice in front of a change board this quarter is dated and narrow: which asset classes lose their testing days, and who signs the exception when a fast patch takes an application down. On the other side of that ledger, Anthropic forecasts that "the risks will only grow as models become more capable" [21].
What to watch
- Whether Anthropic publishes time-to-exploit or cost-per-exploit for the same two patch sets. That is the figure a remediation SLA is actually written against.
- Whether anyone outside Anthropic reproduces the Windows kernel result working from binaries with no source available.
- Whether Mozilla's move to roughly weekly dot releases narrows the median gap below 19 days in the next release train.