Instrumentation Is the Whole Gap Between an Agent and an Agent You Can Run

The eighth installment of a dev.to series called Harness Engineering names observability as the sixth and final component of an agent harness, and separates it from the other five on a specific ground: every other component does something the agent needs, while this one does something the engineer needs ^[1][2][3]. That distinction is the useful part, because it explains why instrumentation keeps getting deferred, and why deferring it leaves you holding a bet you cannot check ^[2].

The author's definition is concrete rather than architectural: logs of each model call with inputs, outputs, latency and token cost; traces of each tool execution recording which tool ran, with what arguments, with what result, and in what order; latency and token metrics tracked over time; and evals, meaning a fixed set of tasks you run the agent against to score behavior and catch regressions ^[4].

The reason ordinary request logging does not cover this is arithmetic, not philosophy. The piece identifies three properties that make agents hard to operate: they are non-deterministic, so the same input can produce different outputs on different runs; they are multi-step, so a single task can involve dozens of model calls and tool invocations; and they run for minutes or hours, long enough that you cannot watch every step live ^[5]. Stack the second property on a per-request logging scheme and one unit of work scatters into dozens of unrelated records, which is why the series treats session-level tracing as a distinct requirement rather than a nicety ^[13]. Stack the first on top and the failing run may not reproduce, which makes the log of that run the only evidence that will ever exist ^[14].

The failure mode the author uses is worth quoting for its ordinariness: an agent burns $12 in tokens and returns a plausible-looking but wrong result, and with no logs there is no way to tell whether the cause was a bad tool call, a bad retrieval, a bad prompt, or the model simply getting it wrong ^[9]. Note what is missing from that list: nothing exotic. Four routine causes, indistinguishable without instrumentation.

The forward-looking argument is the stronger one. You cannot improve a harness you cannot measure, and if you change the system prompt, reshape the tools, or swap in a new model with no before-and-after comparison on a real set of tasks, you are not iterating ^[10]. That is the sentence most teams should test themselves against, because it turns evals from a research artifact into a change-management requirement.

Two caveats on the source. It is a single author writing a series that also sells a Udemy course and a live Maven workshop, both described as optional ^[11]. And the text available breaks off partway through the logging section, after establishing that model calls should be logged with full context sent, full response, latency and token counts broken out by model tier, and that tool traces need arguments, results and ordering ^{[6][7][8][15]}. So the operationally awkward questions, chiefly how long you retain full prompt-and-response payloads and what belongs in a fixed eval set, are not settled in the material at hand ^[15].

What to watch: whether Parts 9 and 10, on harness architecture and decomposing Claude Code, put numbers to the storage and eval overhead this design implies ^[12]. Full-fidelity logging of every context window is a real cost line, and a series that has been specific about mechanism so far will be judged on whether it is equally specific about that.