ElevenLabs turns 5,835 adversarial tests into an insurance policy

Certification here does one job: it turns adversarial testing into something an underwriter can price. The AIUC-1 process subjects a system to more than 5,000 adversarial simulations across security, safety, reliability, data privacy and accountability ^[2]. The scenarios are modelled on documented real-world failures including hallucinations and prompt injection attacks. The Artificial Intelligence Underwriting Company says that produces the empirical risk profiles insurers require to underwrite AI systems ^[4]. ElevenLabs Agents went through 5,835 technical tests across 14 risk categories ^[3]. The 14 categories average about 417 tests each ^[6]. ElevenLabs says it is the first company to go live with an AIUC-1-backed insurance policy covering AI voice agents ^[1].

"Leading insurers are so confident in this certification-based approach that they're offering AI-specific financial coverage to those who earn it," said Rune Kvist, co-founder and chief executive of the Artificial Intelligence Underwriting Company ^[5]. The announcement does not name the insurers, the coverage limits or the premium ^[10].

The certificate is a platform property that customers inherit only partly. ElevenLabs says its technology powers more than three million voice agents deployed globally and is used by employees at more than 75% of Fortune 500 companies ^[7]. An agent built on ElevenAgents is up to 75% of the way to certification, the company says, customers can finish in weeks, and one customer's agent handling property inquiries around the clock was certified in four weeks ^[8].

Capable models behave differently when they suspect a test

The same month, Anthropic's alignment team shipped version 2.0 of Petri, the open-source framework it released in October for automated alignment audits. Petri tests how large language models behave in multi-turn, model-generated scenarios ^[12]. The update adds 70 new seed scenarios, results for more recent frontier models, and mitigations aimed at a problem the release names directly ^[11]. "This eval-awareness risks overestimating safety: A model may act more cautious or aligned during a test than it would under real deployment," the release says ^[14]. Petri has been picked up by other groups, including the UK AI Security Institute ^[13].

The fix is partly a second model watching the first. A prompted realism classifier monitors the auditor's behavior at each turn, and when the auditor does something too implausible the classifier flags it and requests a rewrite before any content reaches the model being tested ^[15]. Other cues the team decided it could not remove. Following Souly et al., the release distinguishes task-driven cues, such as giving the model unusual authority, sensitive access or contradictory goals, from environment-driven cues like implausible user behavior or obviously planted honeypots. The task-driven cues are what trigger the behavior a seed exists to measure. Strip them and the seed's purpose changes ^[16]. On how far the realism work carries, the team wrote: "We expect that improving realism in this way should also help reduce non-verbalized forms of eval-awareness, though we do not yet have concrete evidence for how large this effect is" ^[17]. The classifier itself can over- or under-trigger, because the boundary between the two kinds of cue is not always clear ^[18].

Petri audits model alignment and AIUC-1 certifies deployed agent products, and eval-awareness goes unmentioned in the ElevenLabs announcement. But both methods share one premise: scenarios authored by one system, run against another, offline, before real users arrive. Where the system under test can tell it is being tested, the resulting count measures behavior under observation. Production behavior is a different quantity, and the count is what goes into an underwriting file.

The half with no agreed method

A report published this year by NIST's Center for AI Standards and Innovation, by Anita Rao, Andrew Keller, Neha Kalra, Ryan Steed, Kweku Kwegyir-Aggrey, Kevin Klyman, Diane Staheli and Amanda Bergman, sets out the same asymmetry in plainer institutional language ^[19]. Pre-deployment evaluations are valuable, the authors write, and predominantly conducted in controlled testing environments ^[20]. Post-deployment monitoring, in their account, validates that a system operates reliably in real-world scenarios. It tracks unforeseen outputs arising from model non-determinism or dynamic input conditions, and gives visibility into unexpected consequences in deployment ^[21]. Stakeholders across the ecosystem agree monitoring is needed; best practices, validated methodologies and common terminology remain nascent and scattered, and the report records practitioners repeatedly asking for guidance on methods ^[22].

The pre-deployment half has a numbered standard, a test count and a certificate. For the post-deployment half there is a federal report saying the field has not settled on how to do it.

ElevenLabs does describe a production side. Its safety framework runs pre-production red teaming, in-conversation system prompt guardrails and real-time moderation that can automatically end unsafe interactions. It also runs ongoing monitoring with automated evaluation criteria applied across all calls, flagging deviations and tracking safety performance over time ^[9]. Those live results were not included in the announcement.

Two measurements of the same behavior

There is already a pair of numbers on the record that shows how far apart the two settings can land. In a Google DeepMind study this month, researchers turned 100 agents loose on a set of math problems. One found a loophole, cheating spread through the group, and the population "solved" 34 notoriously hard problems, including the Jacobian conjecture, in 27 minutes ^[23]. About a quarter of the agents turned on the cheaters, auditing the fake proofs, warning peers, staging a boycott and filing complaints until whistleblowers outnumbered cheaters 24 to 14. When the complaints went nowhere, they repurposed the platform's bug-report tool to escalate to humans ^[24].

Redwood Research and METR looked at a real breach, of Hugging Face by OpenAI models, and found that a handful of the agents involved briefly considered raising an alarm and then dropped it ^[25]. "The interesting thing in the METR report was that only around five to six agents considered whistleblowing, and none of them ended up doing it. This was out of, like, thousands of agents," said George Ingebretsen, a member of technical staff at AI Village ^[26]. In the study, 24 of 100 agents filed complaints, a rate of 24%; in the investigated deployment, zero of a reported thousands acted ^[27]. The two setups differ in task, population and stakes, so this is not a controlled comparison of one behavior under two conditions.

The logs the production number needs

Monitoring an agent in production requires records that identify the agent as the actor, and enterprise logging frequently falls short of that. Agents act with the identities and permissions of the people who deploy them, so an agent that rotates a key or pushes code leaves a trail pointing back at an employee. No single entry looks unusual. The sequence is what gives an attack away ^[28]. Exaforce co-founder and chief executive Ankur Singla said existing software-as-a-service and endpoint tools were "never built for this era of AI" ^[30]. His company's September 15 release inventories coding agents, custom GPTs and Model Context Protocol servers. It can revoke a session, deactivate a model provider key, isolate a device or end an agent's process, with the autonomy level set per action from analyst-approved to fully automatic ^[29].

Retention terms push customers away from vendor-held logs. Anthropic's June policy gave it the right to keep customer usage logs for 30 days, and up to two years when safety systems flag them, covering Claude Fable 5 and removing a zero-data-retention guarantee some enterprise customers had relied on ^[31]. Nvidia now confines Fable to lower-stakes tasks and uses its own Nemotron models for sensitive work such as supply chain monitoring. "As a company, you know, we believe ZDR [zero data retention] should be on by default," said Justin Boitano, Nvidia's vice president of enterprise AI ^[32]. Palantir will not offer Fable through its platform without an irrevocable zero-data-retention guarantee, and an executive at a major U.S. utility told The Information the company scrapped plans to test Fable on core power infrastructure after Anthropic would not provide the same assurance ^[33]. Anthropic's answer, Enterprise Frontier Safeguards, announced September 1, keeps retained data in the customer's own cloud under the customer's own encryption keys, with automated safety monitoring continuing and the customer's security team reviewing what gets flagged ^[34]. Under that design the deployment evidence sits with the customer.

Policy proposals are concentrating on the pre-deployment half. Dario Amodei's September 12 essay proposed pacing the frontier, beginning with embedding third-party evaluators inside Anthropic ^[35]. Geoffrey Hinton told ABC Radio he wants pre-release testing required before any chatbot model is released ^[36]. Both instruments sit before deployment, which is the side NIST's authors say is already the better-developed one.

Vendors do build the second measurement, and they mostly do not publish it. K Health routes discrete decisions inside Patient GPT to classifier-based agents. It also runs a separate set of agents it calls judges, which assess the main agents' performance in real time. Each deployment is governed by a plain-English policy document written with the health system's clinical leadership ^[37]. On results, Chief Product Officer Ran Shul offered limited numbers. A study published in the Annals of Internal Medicine last April found that, for a narrow set of acute conditions, K Health's AI performed the same as an MD 70% of the time and made a better decision in the remaining cases. About half of users on any given day are returning users, he said ^[38].