Skip to content

Build1 publisher3 min readPublished

The 2-token prompt that billed 39,966 tokens: what a CLI subprocess hides

A developer avoided API-key coupling by shelling out to the Claude CLI. The default output format returns text only, so months of per-commit model calls left no usage record at all.

The Engineer · Build desk

Drafted by a language model from the sources cited here and checked against its claim ledger before publication. How we use AISend a correction

What happened

  • The project contains a script, git_commit.py, that turns a staged git diff into a Conventional Commit message by shelling out to the Claude CLI.
  • There is no ANTHROPIC_API_KEY anywhere in the project, on purpose: an early version used urllib against the API directly and broke immediately for anyone running on an OAuth session instead of a raw key. The author describes this plumbing as built two months before the post.
  • Every AI call in the repo goes through a claude -p subprocess instead of a direct API call.
  • The author writes that the decision to route all calls through the CLI subprocess is still right.
  • Because there is no API key, there is no per-key usage dashboard, and therefore no line item to audit.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

A developer went to audit his LLM spend and found there was no invoice to audit. His repo's `git_commit.py` turns a staged diff into a Conventional Commit message by shelling out to the Claude CLI, and with the CLI's default output format the stdout he captures is the commit message and nothing else [1][7].

The repo deliberately contains no `ANTHROPIC_API_KEY`. An early version called the API directly with `urllib` and broke immediately for anyone authenticating with an OAuth session rather than a raw key, so every model call in the project now goes through a `claude -p` subprocess [2][3]. The author still considers that the right call [4]. The second-order effect is that with no key there is no per-key usage dashboard, and so no line item [5]. The script has been making a model call on essentially every commit for several months, and he has never known what one cost [6].

The fix is one flag, `--output-format json`, after which the payload carries `result`, `usage` and `total_cost_usd` [8]. Before wiring it in he ran the cheapest probe available, a four-word prompt: reply with exactly: OK [9]. The usage block that came back: `input_tokens` 2, `cache_creation_input_tokens` 39,966, `cache_read_input_tokens` 0, `output_tokens` 4, `total_cost_usd` $0.2408 [10].

The field named `input_tokens` was the wrong field to read. The billed input sat in cache creation: system scaffolding, tool schemas and session context assembled around the prompt before it went anywhere [11]. That is roughly 19,983 times the nominal input [1], and the actual prompt is 0.005 percent of the billed input tokens [2]. The post calls this five orders of magnitude; it is closer to four [20][1]. The direction is the point: estimating spend as `len(prompt) / 4 * rate` models the 2 and misses the 39,966 [12].

The second measurement is the more useful one for anyone running an agent out of a repo root. In August the author found that a `claude -p` subprocess launched from that root was auto-loading the project's `CLAUDE.md` into every commit-message call, a long block of routing rules for MCP tools that do not exist in that process [13]. He detected it behaviourally: asked the subprocess whether it could see the rules, added `--safe-mode`, asked again [14]. He never measured it, because at that point there was nothing to measure with [15].

Running the same trivial prompt with and without the flag: cache creation 5,770 against 7,803, cache read 34,210 against 35,994, output 4 against 313, cost $0.0459 against $0.0633 [16][17]. That is 3,817 extra input tokens [5] and 37.9 percent more money for an identical answer [4], or $0.0174 per commit [8]. Both runs returned the string OK; the author attributes the extra 309 output tokens to the model working through which of the mandated `ctx_*` routing tools it was supposed to use [19]. Output was about 78 times higher [6]. The injected rulebook did not merely pad the prompt, it changed the generation.

One more thing sits in these numbers. The cold probe cost $0.2408 with 39,966 cache-creation tokens and no cache reads, while the later runs cost $0.0459 with 34,210 cache reads [10][16]. Cold is about 5.2 times warm [3], so a single per-call figure says little without knowing which side of the cache it came from.

Worth watching: whether cost estimators start reporting cache-creation tokens as their own line, since that is where the money was [10][11], and whether other headless CLI wrappers keep defaulting to text-only stdout that drops usage on the floor [7]. The author's account is single-sourced and self-reported, one project, one probe [9][16].

Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories