Two defects had to hold at once for anything to leave the lab. The scenario needed a plausible target, so it used a fictional company name that Irregular believed matched no real entity, and human oversight let that name land on a real domain [4]. Separately, a few interactions with the environments had internet access that was not supposed to be there [3]. Neither does much alone: a name collision inside a sealed environment is a typo, and open egress aimed at a domain nobody owns is wasted packets. Together they hand a model being scored on vulnerability research a live target with an owner [13].
The detail I would raise in review is the wander. Irregular reports that in one instance a model left the intended target for a different site with a somewhat similar name, and found credentials there that someone had posted publicly [6]. Once a process inside the environment can resolve arbitrary names, the scenario's target list stops predicting where the model goes and DNS starts. The check that would have caught the original collision costs one lookup at fixture-authoring time.
The accounting deserves separate scrutiny from the mechanism. Irregular states that all subsequent public disclosures trace back to the issue its customer first disclosed on July 30 and are not materially separate incidents [1], and that the problem came from a single evaluation scenario, was resolved before the first public disclosure, and leaves nothing active today [2]. That is Irregular's read of other organisations' disclosures, published deliberately after their own comments [12], while its deep audit is still running [8]. It also sits beside Irregular's position that internet access standards in pre-deployment evaluation are implicated in many different recent incidents by multiple organizations [7]. One scenario at one vendor is compatible with a class of failure that has more addresses than that.
For any of this to transfer to your own eval stack, you need the denominator. Irregular says the process typically runs at least thousands of simulations across multiple models before a deployment decision, including a baseline model for comparison [10], and describes the offensive consideration as arising in a small fraction of cases [5]. No count is given [14]. A small fraction of several thousand runs is still a set of requests that arrived somewhere real, so the figure worth measuring is not a model misbehaviour rate but the share of your scenarios that can open a socket by default, and how many fixture names were ever resolved before use.
The cost of the obvious remedy is real. Deny-by-default egress per scenario, an allowlist a human signs, and fixtures registered or checked all make cyber-capability evaluation less faithful, because you end up simulating the internet in a test whose whole point is behaviour against the real one. That is the tension the promised whitepaper has to resolve rather than restate [7]. Meanwhile, a post-mortem that names the human oversight and the off-target wander instead of gesturing at unruly model behaviour is the part of this worth copying [4][6].
