Build1 distinct publisher3 min readPublished
Measured per call rather than averaged, one voice agent's outbound cache hit rate came in at less than half of inbound on identical code and prompts, and the cause was two variable fields sitting inside the cached prefix.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
The floor is a diagnostic. A provider that cannot find 1,024 matching tokens caches nothing at all, not a partial hit [5]. So when hundreds of calls in one log record exactly that number, the longest matching prefix ran out just past the floor and the provider kept the smallest slice it is permitted to keep [12]. Prefix matching is byte for byte from the very first token, and it stops at the first character that differs [4]. Put a customer name about a thousand tokens in [13] and every token after it is dead weight, however stable that text is.
That sets a ceiling worth the arithmetic. The team's dev.to writeup says its prompt is several times bigger than the minimum [8]. Take several at three: 1,024 cached out of 3,072 prompt tokens is 33% [17], which is the best those calls could have scored with the name still in place, before anyone argues about server affinity. The reported blend was around 40%, with about a third of eligible calls caching nothing [8], and the split by direction was fifty points [9]. A fifty point gap around a 40% blend puts outbound below 40% and inbound above 50% [16].
Pre-warming is the part I would push on. Firing a throwaway request with the same system prompt before the call connects does work, and the team does it on every call [2]. What it buys is reuse inside one call: turn 2 reuses turn 1, turn 3 reuses turn 2, and once a call has made a couple of requests against the same machine the later turns fix themselves [15]. Turn 1 has no earlier turn in its own call to reuse [15], and most of their outbound calls never get past turn 1 or 2 [11]. The writeup does not say whether the pre-warm request carried that call's customer name. If it did, turn 1 should have matched, and the gap would have to come from the third condition, which is the machine the request landed on [6]. If it did not, the name accounts for the asymmetry on its own. The average was never the number either way.
For 24% and 40% to say anything about your agent, the prompt has to clear the provider floor [5], at least one per-call field has to sit inside the prefix [13], and your traffic has to be weighted toward one and two turn calls [11]. Break that last condition and the problem mostly self-corrects: on a ten turn call the average heals itself, and turn 1 becomes a rounding error on the bill. It is still the turn where the caller is sitting in silence, and prefix caching is worth hundreds of milliseconds of time to first token [14].
The byte-for-byte rule also makes the remedy mechanical rather than clever: everything that varies per call goes after everything that does not [4]. In my context I would ship that ordering change before touching the warm-up, because the reorder is a text edit and the pre-warm is an extra request on every call [2].
Ranked by verification strength, evidence, and original report placement.
Condition one of prefix caching: the prefix must match exactly, byte for byte, starting from the very first token; one character different and the match stops right there.
Condition two: the prefix has to be long enough, and below the minimum, usually 1,024 tokens, nothing is cached at all, not a partial hit.
Condition three: the cache lives in the memory of one server, and while providers route requests on a hash of the start of the prompt, under load they spread traffic around, so landing on a new machine means a cold cache no matter how clean the prompt is.
A voice agent team that measured prompt cache hit rate found its outbound calls doing less than half as well as its inbound calls, with the same code, the same provider and the same prompt template.
The team pre-warms before every call by firing a throwaway request with the same system prompt so the provider caches it before the caller says hello.
When a prompt is sent, the provider processes it into an internal KV cache representation and keeps it for a few minutes; if the next request starts with the exact same text, the provider skips reprocessing that part and only works on what is new.
Distinct publishers with included, body-backed reporting in this cluster.
1 article · September 4, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
Claude's prompt cache dies quietly in agent loops: the 20-block lookback nobody configures1 distinct publisher
build
Compaction that cut tool output 38.4% pushed the bill up 6.8%1 distinct publisher
build
A cached prompt prefix repays its write premium on the second request1 distinct publisher
build
A semantic cache hit saves five times what a prompt cache hit saves1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Unverified: one team's logs, no outside check
Every figure traces to a single dev.to write-up reading its own logs: the direction split, the turn-depth curve, the zero-cache share and the 1,024-token mode. The provider is never named and no documentation is cited, so the hard floor and the load-balancing behaviour stand on the author's word. Working in the account's favour is that its arithmetic is self-consistent, with 1,024 tokens of a 7,200-token prompt matching the 14% reported for single-turn calls.
Adoption claim rests on a single measured deployment
Pre-warming is described as the thing everyone recommends for voice agents, and the team says it does it on every call, but the only deployment with measurements behind it is the author's own. No second team, vendor disclosure or benchmark supports the idea that outbound voice traffic generally behaves this way, and the post gives no call volumes to size even this one case.
Headline banks a result the text never shows
The title sells a doubled hit rate; the published account diagnoses the cause and cuts off before any after-figure appears. The diagnosis itself is soberly handled, with the author killing two of their own earlier theories about batch-dial concurrency and explicit cache keys, so the overstatement is concentrated in the promise rather than the analysis.
Credibility post, but nothing for sale
A fixed-it headline on a developer platform pays its author in reputation, and the opening question about turn-1 hit rate is framed to make readers suspect they are behind. Against that, no product is pitched, no vendor is named and no tool is linked, and most of the length goes to measurement detail that promotional writing would skip.
Mechanism holds up; the magnitudes are unaudited
The causal chain is tight enough to act on. A variable field roughly a thousand tokens into the prompt explains both the exact-1,024 mode in the logs and the flat rate on single-turn calls, and no competing explanation survives in the account. The magnitudes are a different matter: the 40% blend, the fifty point direction gap and the third of calls at zero come with no sample size, no window and no outside check.