Published · 2d agoProduct2 min read
67% On Uber's Own Benchmark Is The Number That Writes Your Agent Buyer Checklist
Uber's ADR detects 67 percent of attacks on its ADR-Bench with zero false positives. The missing third is what moves agent security from detection features to authorization questions.
Not a builder's beat, but builders have a standing stake in it.See today for builders
What happened
- On ADR-Bench (302 tasks, 17 attack techniques, 133 MCP servers), ADR achieves zero false positives while detecting 67% of attacks, outperforming three baselines (ALRPHFS, GuardAgent, LlamaFirewall) by 2-4 in F1-score; the paper was accepted to MLSys 2026 (Industry Track).
- ADR has been deployed at Uber for over ten months, with adoption reaching over 7,200 unique hosts and processing over 10,000 agent sessions daily, uncovering hundreds of credential exposures across 26 categories and enabling a shift-left prevention layer (97.2% precision, 206 detected credentials).
- ADR left about a third of ADR-Bench attacks undetected.
- On AgentDojo, a public prompt injection benchmark, ADR detects all attacks with only three false alarms out of 93 tasks.
- Cloudflare's Agent Access Model proposes making the agent's capability smaller so there is less to judge, rather than making each access decision smarter, and states that controls built for humans fail quietly when pointed at agents, by granting too much, seeing too little, and trusting for too long.
Compiled by The Product DeskSomething wrong?How this is made
Why it matters
The figure comes from the ADR paper accepted to MLSys 2026: Uber's Agentic AI Detection and Response system detects 67% of attacks on ADR-Bench, its own benchmark of 302 tasks, 17 attack techniques and 133 MCP servers, with zero false positives, beating ALRPHFS, GuardAgent and LlamaFirewall by 2 to 4 in F1 score [1]. This is not a lab claim: ADR has run at Uber for more than ten months, reaching over 7,200 unique hosts and more than 10,000 agent sessions daily, and has uncovered hundreds of credential exposures across 26 categories [2].
What the number turns on is the remainder. Roughly a third of the benchmark's attacks were not detected [3], and the same system detects every attack on the public AgentDojo benchmark with three false alarms across 93 tasks [4], so a single detection rate travels badly between test sets. Treat 67% as the honest ceiling of a mature detector and the buyer questions move upstream, to what the agent was allowed to do in the first place.
That is where the three vendor frameworks converge, and where a checklist is available. Cloudflare's Agent Access Model argues for making the agent's capability smaller rather than each decision smarter, because human-shaped controls fail quietly by granting too much, seeing too little and trusting for too long [5]; it holds that credential lifetime should match task lifetime, often minutes [6], that preventive controls must run inline at the point of action because an agent can read a table and POST it outbound before a human-tuned control finishes sampling [7], and that the prompt is not a perimeter [8]. Google's Beyond Zero makes the same move at the decision layer: authorize individual actions on specific resources, uniformly across front ends, APIs and MCP, rather than granting access to a whole application [9]. 1Password's version is operational: issue credentials at runtime, scoped to the task, with authorization that ends when the job does [10], and for delegated agents, prove whose authorisation it holds, since logging actions under a service account fails that requirement entirely [11].
Ask suppliers for their number, the benchmark it came from, and what handles the miss.
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
On ADR-Bench (302 tasks, 17 attack techniques, 133 MCP servers), ADR achieves zero false positives while detecting 67% of attacks, outperforming three baselines (ALRPHFS, GuardAgent, LlamaFirewall) by 2-4 in F1-score; the paper was accepted to MLSys 2026 (Industry Track).
- [2]
ADR has been deployed at Uber for over ten months, with adoption reaching over 7,200 unique hosts and processing over 10,000 agent sessions daily, uncovering hundreds of credential exposures across 26 categories and enabling a shift-left prevention layer (97.2% precision, 206 detected credentials).
ReportedView cited source - [4]
On AgentDojo, a public prompt injection benchmark, ADR detects all attacks with only three false alarms out of 93 tasks.
ReportedView cited source - [5]
Cloudflare's Agent Access Model proposes making the agent's capability smaller so there is less to judge, rather than making each access decision smarter, and states that controls built for humans fail quietly when pointed at agents, by granting too much, seeing too little, and trusting for too long.
ReportedView cited source - [6]
Cloudflare argues the lifetime of the credential should match the lifetime of the task, which for an agent is often minutes, because service account credentials designed for long-lived software outlive the work they were issued for and remain in memory, logs or environment variables where they can be replayed.
ReportedView cited source - [7]
Cloudflare notes that an agent with a database connection and an outbound network path can read a table and POST it to an external endpoint before a human-tuned control has finished sampling, so preventive controls have to run inline, at the point of action.
ReportedView cited source
Sources & coverage · 5 publishers
The reporting this story was synthesized from, earliest first. Every link goes to the original.
- blog.google2d agoGoogle introduces Beyond Zero for AI enterprise security
- blog.cloudflare.com2d agoThe Agent Access Model | Cloudflare Blog
- 1password.com2d agoRemove standing access before AI agents exploit it | 1Password
