An attack harness closed 67 points of Booz Allen's own AI threat ranking

The tab is usually called Approved Models, and somewhere to the right there is a column marked risk tier. Booz Allen's table fills that column in nicely: Grok-4.5 on 49, GPT-5.6 Sol on 46, Meta's Muse Spark 1.1 and Moonshot's Kimi K3 tied on 38, Alibaba's Qwen3-Coder last on 4 ^[8]. The firm's own conclusion is that the column is measuring the wrong object, because the model is no longer the unit of risk ^[13].

The test was built to produce exactly that column and no more. Each model sat on its own attacker machine and issued commands one at a time, with no tool menu and no supporting software, because Booz Allen wanted the model stripped of the engineering that normally surrounds it ^[3]. It then scored what the traffic records and intrusion-detection sensors proved rather than what the models claimed to have done ^[5], which is more than most benchmarks bother with. What that design cannot tell you is what someone with a laptop and a fortnight would build around any of these.

Sonnet 5 came 15th of 18 on a score of 13, and rose to rival Mythos once Booz Allen attached an attack harness, the software that wires a model to hacking tools and keeps it on task ^[10]. Booz Allen puts the closed gap at 67 points ^[11]. Run it as a ratio instead and 13 becomes roughly 80, a factor of about six ^[1]. Either way, the swing from adding plumbing is bigger than the 45 points that separate second place from last ^[2]. The variable that moved the number most is not on the leaderboard at all.

An attacker takes whatever model is available and unlikely to refuse, then spends the effort on the harness, because the harness is the part that keeps the model focused, lets it recover from failure and chains single actions into a sustained operation ^[12]. Security teams, by contrast, tend to reach for the top of the ranking. The refusal finding points the same way. One model declined a job for want of credentials, and its cyber-tuned sibling took the identical job and completed it ^[15], which is why Booz Allen treats guardrails as a function of context and configuration rather than a property of a model ^[16].

None of this comes out tidy, though, for two reasons. Booz Allen concedes it has not tested Chinese or open-weight models with optimised harnesses, while saying its results strongly suggest fully capable combinations already exist ^[14], so the cheapest dangerous pairing carries no score. And on real-world vulnerability research the report says all nine frontier models scored zero against an unseen production flaw, in the same breath as saying Anthropic's frontier models spotted it and Mythos could exploit it ^[18][19]. The Next Web reads the zero as an artefact of scoring ^[19]. Anyone banking the breathing-room framing, and Booz Allen's expectation that most of the 18 will catch up to Mythos ^[21], is banking on that reading rather than on the number.

The risk-tier column, then, needs two entries per row rather than one. First, what the model did alone under log-verified conditions. Second, whether anyone has measured it wrapped in the smallest harness a competent adversary would reach for. Where the second is blank, the row is unscored rather than low risk. Sonnet 5's 13 measured a bare configuration, one stripped of the harness that turns a model into an operation, and no attacker builds it that way.