Topic
Instrumenting LLM agents with logs, session traces, latency and token metrics, and evals so runs can be inspected and debugged after the fact.
Anji Xu's open-source Claude Statuspane puts context use, five-hour and seven-day rate limits and session cost in a card above the Claude Code prompt. The readout lives on that one developer's screen, so whoever answers for a team's total agent spend still needs records kept somewhere central.
Reality
LiteLLM launched Lens on September 30, a tool that uses AI agents to find recurring failures across agent traces sent through its model gateway. Customers host the analyzer and its databases, and the 200,000-trace volume CTO Ishaan Jaffer cites is a future target Lens has not been measured against.
Reality
Strands and CrewAI agents exited 0 on all six runs where a broken verification tool checked nothing, a recorded-proxy test on dev.to found. Only tool-call traffic captured outside the framework showed the failures.
Reality
AWS made CloudWatch Omni generally available with 17 built-in evaluators that score an AI agent's answers on live production traffic. For operators, the job moves from confirming an agent is running to deciding whether an automated score is good enough to gate a release.
Reality
One OpenAI Codex prompt spawned 826 child agents and burned about $78,000 in credits, according to the user's own reconstruction. Nearly all the counted tokens trace to an alpha client build, and only OpenAI's servers can turn them into dollars.
Reality
LangChain's survey of 1,340 practitioners found 89% had observability on their agents and 37.3% ran online evaluations. The OpenTelemetry GenAI attribute registry explains why only the second number measures quality.
Reality
Microsoft's public preview flags jailbreaks, prompt injection and credential leakage from agent telemetry, and agents built outside Copilot Studio, Foundry and Agent Builder need the Agent 365 SDK first.
Reality
AWS has added three skill-focused evaluators to Strands Evals and Bedrock AgentCore Evaluations. Two of them call a model once per invoked skill, and the third is a deterministic check that exists only in Strands.
Reality
Microsoft Foundry writes agent runs as OpenTelemetry spans with gen_ai attributes and exports them to Azure Monitor, where evaluators score sampled production traffic off the same records. Those attributes include tool arguments.
Reality
The MIT-licensed TypeScript framework at v0.16 exposes seven named run phases with hooks on each side, and its case for harness over model rests on one incident-triage run that a 4B local model and Claude both finished.
Reality
shinpr has taken claude-code-workflows through 133 releases, and the recent ones delete structure the models no longer need. His session reader then found three mandatory steps in his own repository that never ran.
Reality
Token Meter reads the trace files Claude Code, Codex and Cursor already leave on a developer's disk and prices them against published model rates. The budget alert it fires goes to whoever ran the session.
Reality
Warp's case for LLM-as-a-judge scoring is that a coding agent leaves a complete record you can grade after the fact. Each dimension gets a prompt, a rubric and a judge model of its own, and it runs at about 3% of the company's own token bill.
Reality
The $35 million Series A funds a bet that a failure caught in production can be rerun as a pre-merge test, and making that work means standing up stateful fakes of every database, payment system and API the agent touched.
Reality
The private preview reads OpenTelemetry traces from agents on Gemini Enterprise, files findings into Security Command Center with a severity and a plain-language rationale, and only sees agents already instrumented to its spec.
Reality
Arcjet's new runtime security takes agent activity in through the OpenTelemetry pipelines platform teams already run, then evaluates each tool call against Open Policy Agent rules before it executes and again after.
Reality
A developer instrumented 89 coding sessions to test whether rules the agent had already read changed what it did. Enforcement only started working once the rule was rewritten into something a hook could see.
Reality
A no-code support agent called an endpoint it invented, was refused twice, and still told the customer it could see her charges. Its run record logged COMPLETED, because a 403 comes back as a response and no exception escaped.
Reality
The engineering lead at app platform GoodBarber traced 70 duplicate paragraphs to a read path serving a 742-byte empty list out of a 60-second cache. His debugging order now puts the prompt last.
Reality
An AWS post on agent monitoring describes failures that return clean responses and throw no exceptions, and answers them with a judge model that scores live interactions for helpfulness, correctness and goal completion.
Reality
Build · September 10, 2026 · 1 publisher
Build · September 2, 2026 · 1 publisher
Build · September 1, 2026 · 1 publisher
Build · August 31, 2026 · 1 publisher
Build · August 30, 2026 · 1 publisher
Build · August 29, 2026 · 1 publisher
Build · August 28, 2026 · 1 publisher
Build · August 26, 2026 · 1 publisher
Build · August 26, 2026 · 1 publisher
Build · August 24, 2026 · 1 publisher
Build · August 24, 2026 · 1 publisher
Build · August 24, 2026 · 1 publisher
Build · August 23, 2026 · 1 publisher
Build · August 23, 2026 · 1 publisher
Build · August 23, 2026 · 1 publisher
Build · August 22, 2026 · 1 publisher
Product · August 19, 2026 · 1 publisher
Build · August 19, 2026 · 1 publisher
Build · August 19, 2026 · 1 publisher
Build · August 15, 2026 · 1 publisher
Build · August 15, 2026 · 1 publisher