Build1 distinct publisher3 min readUpdated
A framework running at Uber for over ten months argues endpoint tooling is structurally blind to agent reasoning. On the authors' own benchmark it still misses a third of attacks.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
Follow any of these and your For You feed starts watching them — no settings page required.
A team of Uber-affiliated authors and academic collaborators has published the design and production record of ADR, a detection and response system for AI agents operating through the Model Context Protocol, carrying a copyright line for the MLSys Conference Industry Track in Bellevue, Washington, 2026 [1]. Their central claim is structural rather than incremental: endpoint detection and response tools can observe a file write or a network call, but not the user prompt, the agent's reasoning steps, or the causal chain that links intent to that execution [2].
That is a real gap, and it is not one you close with tuning. MCP is a standardised interface for connecting large language models to external tools and data sources [3], and the paper argues it opens an attack surface where agents can be manipulated through natural language, exploited via compromised MCP servers, or coerced into running unsafe commands and exfiltrating data [4]. An EDR alert on a suspicious write tells you a process misbehaved; it does not tell you whether a poisoned tool description talked the agent into it. The authors also fault static guardrails and rule-based policy checkers for failing to generalise across attack techniques and enterprise contexts [5], and flag severe class imbalance as a further obstacle [6].
The third constraint is the one operators will feel first: the paper states that LLM-based inference is prohibitively expensive at scale [7]. ADR's answer is a three-part split. A lightweight endpoint sensor reconstructs high-fidelity agentic telemetry [8]; an "Explorer" component does pre-deployment red teaming and hard-example generation [9]; and the detector runs two tiers, fast triage in front of context-aware reasoning [10]. In other words, you do not get to put a reasoning model on every session, so you build a cheap filter and reserve the expensive judgement for what survives it.
The deployment numbers are the most interesting part, because they are operational rather than benchmark. ADR has run for more than ten months across over 7,200 unique hosts, processing more than 10,000 agent sessions daily [11]. It surfaced hundreds of credential exposures across 26 categories, and the authors report a shift-left prevention layer at 97.2 percent precision with 206 detected credentials [12]. Credential leakage, not exotic agent hijacking, is what the telemetry actually found at volume.
The benchmark results are more mixed than the framing suggests. On ADR-Bench, which the same team introduces with 302 tasks, 17 techniques and 133 MCP servers, ADR reports zero false positives while detecting 67 percent of attacks, which the paper says beats ALRPHFS, GuardAgent and LlamaFirewall "by 2-4 in F1-score" [13]. Zero false positives is worth something; missing roughly a third of attacks on your own benchmark is worth remembering [14]. On the public AgentDojo prompt injection benchmark it detects all attacks with three false alarms across 93 tasks [15], about 3.2 percent of tasks [16].
Two caveats. These are self-reported figures from a single enterprise deployment, and the ADR-Bench scoreboard was built by the party being scored [13]. The version available is an arXiv rendering carrying automated warnings that its page layout violates the ICML style [17].
Worth watching: whether anyone outside Uber reproduces the 67 percent figure, whether the 97.2 percent precision on credential prevention holds as agent traffic grows past 10,000 daily sessions, and whether ADR-Bench is actually released in a form competitors can run.
Ranked by verification strength, evidence, and original report placement.
The Agentic AI Detection and Response (ADR) system is presented by authors including Chenning Li, Pan Hu, Justin Xu, Mohammad Alizadeh, Pengyu Zhang and Ming Zhang, with a copyright line for the Proceedings of the MLSys Conference (Industry Track), Bellevue, WA, USA, 2026; the authors describe it as the first large-scale, production-proven enterprise framework for securing AI agents operating through the Model Context Protocol (MCP).
The paper names severe class imbalance, alongside limited observability and poor generalization, as making existing mechanisms insufficient for securing AI agents at enterprise scale.
The authors introduce ADR-Bench (302 tasks, 17 techniques, 133 MCP servers), where ADR achieves zero false positives while detecting 67% of attacks, outperforming three baselines (ALRPHFS, GuardAgent, LlamaFirewall) by 2-4 in F1-score.
The paper identifies limited observability as a persistent challenge: existing Endpoint Detection and Response (EDR) tools see file writes and network calls but not the agent reasoning, prompts, or causal chains linking intent to execution.
MCP is described as a standardized interface for connecting large language models to external tools and data sources, used by enterprises to deploy agents that analyze documents, modify infrastructure, generate code and interact with internal systems.
The paper states that agents can be manipulated through natural language, exploited via compromised MCP servers, or coerced into executing unsafe commands and exfiltrating sensitive data.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Detailed but wholly self-reported, single-source
The supplied material is one arXiv preprint carrying specific, checkable-looking numbers (ten months, 7,200 hosts, 10,000 sessions/day, 26 credential categories, 97.2% precision, 302-task benchmark, 67% detection, 3/93 AgentDojo false alarms) and a clear architectural description. That specificity lifts evidence above anecdote. What holds it below the midpoint is that every figure originates with the team that built, deployed and benchmarked the system, the primary benchmark is the authors' own construction with only 42 malicious tasks, no independent replication or third-party audit appears, and the rendered page is a preprint build flagged by arXiv's automated style checker rather than a published proceedings version. Overhead, latency and cost figures that would let an outsider stress-test the cost argument are not in the supplied text.
Real production scale, but one enterprise
Adoption is genuinely operational rather than aspirational: ten-plus months in production, over 7,200 unique hosts, over 10,000 agent sessions processed daily, and a downstream prevention layer that has acted on 206 detected credentials. That is materially more than a pilot. It is capped by breadth: all deployment evidence comes from a single enterprise (Uber), which is also the authors' own environment, with no disclosed external users, no evidence anyone outside the team has run ADR-Bench, and no license, pricing or availability signal indicating the system can be adopted elsewhere.
Superlative framing outruns the detection numbers
The paper's own results are reported plainly, so this is a moderate rather than severe gap. It is positive because the framing claims more than the measurements deliver: 'first large-scale, production-proven enterprise framework' is a priority superlative with no comparative evidence, and the abstract foregrounds 'zero false positives' and 'sustained reliable detection' while the same sentence's 67% recall means about a third of the authors' own curated attacks are missed - a limitation the supplied text never names. The near-perfect AgentDojo figure sits in the same breath as the weaker in-house result without distinguishing the narrower scope of the public prompt-injection suite. Working against a larger gap: the production numbers are specific and unembellished, and the cost and observability arguments are concrete.
Author-built system, author-built benchmark, author-run evaluation
The incentive structure is visible on the page rather than inferred. The same team designed ADR, deployed it in its own enterprise, constructed the benchmark used to score it, selected the three baselines it is compared against, and asserts a first-of-its-kind priority claim in a conference industry track - a venue where publishing doubles as institutional and professional positioning. There is no adversarial reviewer, competing vendor, or independent evaluator in the cluster to discipline any of those choices. This is not evidence of misreporting; it is a reason to weight the self-reported detection and precision figures cautiously until reproduced.
Claims well-pinned to text, but only one text exists
Confidence in what the paper says is high - the abstract and introduction are explicit and the numbers are unambiguous, so the claim ledger maps cleanly onto locatable statements. Confidence in whether the world works as described is much lower: one publisher, one source, one enterprise, one self-authored benchmark, no replication, and a preprint rendering flagged by arXiv's own style checker. The composite sits below the midpoint because the assessment can verify the assertions but not the underlying performance.
build
Kubernetes MCP servers hide the delete tool; hiding is not removing1 distinct publisher
build
The only way to prove a contract test can fail is to ship a server that lies1 distinct publisher
build
MCP 2026-07-28 drops the `result` wrapper, and your unit tests will not notice1 distinct publisher
product
Twin1's $20M bet: the unit of enterprise AI is one employee, not the org1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
1 article · August 21, 2026