The useful artifact in the dev.to writeup is the field list, not the anecdote. Every span, the author says, carries duration, token counts, cost, model name, and the input and output that passed through it [10]. Set that against the four things the logistics team went looking for and did not have: what the retrieval step returned, what the prompt looked like that day, which model version answered, and how many tokens it burned to be wrong [6]. All four sit on that span attribute list [2]. This is a schema gap. The library recorded the transport faithfully, and the wrongness was inside the payload.
Follow the chain and you can see why nothing fired. Retrieval returned an empty chunk and the prompt went out truncated; the model answered anyway [11]. An empty result list is still a valid result list, and a truncated prompt is still a valid prompt. Neither raises a status code, and the fluent answer that follows returns a 200 with normal latency [3][12]. Hundreds of runs a day [2] across three weeks [1] puts a floor around 2,100 answers behind that gap, reading "hundreds" as the smallest thing the word can mean [1].
A trace only tells you what broke. The author's second half is the part I would argue about at review: a regression set of 50 to 100 de-identified real interactions [15], scored on exact checks and rubric checks whenever a prompt, retrieval strategy or model changes [16], with a rule that a three-point drop blocks the ship even when the change fixes reported incidents [18]. For that gate to mean anything, run-to-run variance on an unchanged set has to be smaller than the gate. At 50 cases each flipped case moves the pass rate two points, so a three-point drop is at least two cases; at 100 cases it is three [3]. Given that the same prompt can be right at 9:00 and hallucinate at 9:15 with no code change [7], the first experiment is running the unchanged set twice and measuring the spread. If the spread is four points, the gate is measuring noise dressed up as rigor.
Adoption cost is real, and it is not CPU. The logging layer keeps the full prompt and completion for every run, plus retrieved chunks with their scores and sources, tool arguments and results, and a stable run ID [13]. That store grows with tokens rather than with requests, and it also retains a searchable record of customers' addresses.
In my context I would ship two span attributes before the agent talks to a customer: the retrieval payload and the model version. Those are the two the incident narrative turns on. Token counts come third, because a retry loop can multiply the bill tenfold without throwing an exception [8] and request counters sit flat through all of it. Cost per resolved task and cache hit rate [14] are good numbers, but they help you tune the system rather than catch failures like this one.
