$0.27 a turn: the context window is a capacity, and somebody is paying for the rest

Start with the arithmetic the dashboard did not show. At roughly $0.50 per million input tokens, 55,000 redundant tokens is about 2.75 cents of pure re-reading per turn ^[9]. Across a queue handling 1,200 conversations a day, that is roughly $33 a day if each conversation is a single turn ^[10] - and the conversation in the logs ran to 40 turns ^[3]. The author's reported figure of $0.27 per turn against an expected few cents ^[2] is consistent with that: the waste is not a line item you can find, because it is distributed evenly across every call your application makes.

The mechanism is boring, which is why it survives code review. The model is stateless between requests; whatever looks like memory is a storage-and-injection strategy the application performs ^[6]. So the default that requires no design work is to append. Append the user turn, append the assistant turn, send the lot. Nothing in the response tells you it was wasteful, because every token in the window is processed on every forward pass, and cost and latency rise with the window at minimum linearly ^[5]. A full window is the most expensive state the system can occupy ^[5]. There is no error code for expensive.

The tutorials' framing that a big window means you can just send everything ^[4] is a claim about capacity that gets read as a claim about correctness. The author's distinction is the useful one: the window is a container, and memory is what you decide to put in it ^[1]. That decision has a price attached, which makes it a billing decision that gets filed under architecture.

There is a second cost that does not appear on any invoice. Benchmarks show the lost-in-the-middle behaviour: models handle a needed fact well at the start or end of a long input and much worse when it is buried in the middle, and larger windows reduce but do not remove the effect ^[7]. So the appended transcript is not neutral padding. It is padding that pushes the relevant turn into the least reliable region of the sequence while you pay for it. The author's framing of working memory as the place where cost and forgetting collide ^[8] is the honest description: more history means more spend and a weaker thread, less history means lost context.

The fix in the source is not clever, and that is a point in its favour. Keep the full record of past turns in a database, Postgres or Redis, and project only a curated slice into the window when it is needed ^[11]. The failure mode in the retail agent was shipping that record straight into working memory uncurated ^[12]. Nothing about that requires a vector store, an embedding pipeline, or a memory product. It requires someone to own the question of what earns a seat in the container, and to answer it in code rather than by default.

One caveat on the numbers: this is a single engineer's account of one client system ^[2], with a per-token price and a daily volume that belong to that deployment ^[9][10]. The ratio is what travels. If your agent appends, your marginal cost per turn grows with conversation length while the answer quality does not, and the only reason you have not noticed is that the window was large enough to hide it.