Build1 publisher3 min readPublished
A production agent's LLM judge filed 113 of 114 state defects under brand voice
Exhaustive human review of a deployed food ordering agent found that its judge writes the defect down and then scores it on the wrong axis. The shipping gate trips only on hangs and hard assertions, so 100 rounds passed clean.
The Engineer · Build desk
What happened
- Researchers measured a deployed per-tenant ordering agent for South-East Asian food and beverage brands against exhaustive human review of its multi-turn transcripts as ground truth.
- In one review batch the agent's built-in LLM judge auto-captured 2 of the 9 systematic problem patterns that human reviewers identified.
- The operational gate flagged zero of 100 rounds in a batch where human transcript review confirmed 23 distinct defects and promoted 7 new cross-cutting patterns.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- contradiction The taxonomy says the judge misses cross-turn defects, while the mechanism section shows its notes described 114 of them, so the failure is in the filing.
- constraint A green dashboard from this gate cannot be turned into a defect-rate estimate, because at an apparent rate of zero the correction the paper uses has no signal to scale.
- cost The instrument that actually found the defects was exhaustive human transcript review, so a team that wants this coverage pays reviewer hours per batch.
- exposure The defect classes that survive the gate are the cross-turn ones: cart hallucination, confirm-gate lockout, escalation lockout and stale referents reach live traffic unflagged.
Call the judge on one round and you get two artefacts: a free-text note, and scores on three axes, intent, brand voice and personalization [5]. The note is often right. Across the corpus, 114 rounds carry a raw judge note describing a confirm-gate or cart-state defect, and 113 of them are scored on the brand-voice axis [6], about 99 percent [2]. Exactly one was filed anywhere else [3]. None of the 114 became an operational failure, because the shipping gate is wired to hangs and hard assertions and never reads the rubric [7].
So the rubric has a place to file a fabricated statistic and no place to file a cart that lost its contents [5][9]. The authors call this "a routing-and-wiring failure, not blindness" [8].
Add rubric categories for state-tracking, guardrails and recovery, the dimensions where the confirmed defects cluster [5]. Then give a bad rubric score a path to failing the gate; today only hangs and hard assertions do that [7]. A stronger judge would only improve the wording of a note that nothing downstream consumes.
The reported recall needs handling with care. Across three batches the judge surfaced well under a quarter of human-confirmed systematic problems [2], and the one legible batch figure is 2 of 9, about 22 percent [3][1]. The posted abstract shows empty parentheses where the percentages for the other results belong, so those figures are not readable in the HTML [17].
For 2 of 9 to transfer to your stack, two things about your setup would have to match. Your rubric axes would have to be turn-local, with no state-tracking bucket [5]. Your gate would have to trigger on process signals only [7]. If your gate already fails on a rubric score, the wiring half of the finding does not apply to you; the taxonomy half still does, since recall in this corpus degrades monotonically with how cross-turn the problem is [10]. The authors say their findings are "generalizable in shape if not in exact value" [11].
The prevalence numbers are where the zero does damage. That batch's apparent defect rate is 0 of 100 rounds, or 0 percent [4], against 23 defects human review confirmed in the same 100 [4]. Where the gate does report a nonzero rate, the paper says the same estimator implies an order-of-magnitude undercount at the measured sensitivity [13].
Low judge recall on single-turn error detection was already reported by Kamoi et al. in 2024, and Lu et al. published a benchmark that judges the judges of agents in 2025 [16]. This paper adds the gate. Reliability for LLM judges is almost always reported as agreement with human ratings, not as recall of real defects [15], and the paper's conclusion is that for production multi-turn agents automated judging is "a regression floor, not a substitute for human review" [14].
What to watch
- The two batch recall percentages missing from the posted abstract would pin how far below a quarter the judge's recall actually sits.
- A gate rewired to fail on rubric scores, re-measured against the same corpus, would show how much of the miss was routing.
- A replication on an agent outside per-tenant food-and-beverage ordering would test the authors' claim that the shape generalizes.