A 12MB Go binary bets agent cost control is cache stickiness, not a dashboard

A solo maintainer has published the design reasoning behind VMR, a local single-binary LLM router written in Go that ships with no database, no web UI and no Node or Python runtime ^[1][2]. The reason to read it is not the minimalism but the argument underneath: that the cost lever for an unattended coding agent is session-sticky cache affinity plus byte-faithful pass-through, and that the author is willing to publish overhead numbers to defend that ^[4][5][7].

Two incidents, as described by the author, set the constraints. The first was a bill: after wiring Claude Code and Cursor to several providers by hardcoding a Base URL per provider, the approach fell apart because agent sessions are long-context and high-frequency, and the moment a subagent switch slightly perturbs the request body the provider's prompt cache key misses and cost climbs visibly ^[8][9]. The second was an outage: a batch refactor meant to run roughly an hour unattended died overnight after an upstream key hit 429 and the proxy wedged, losing the progress ^[10]. The stated conclusion is that the proxy layer must be self-healing, self-retrying and evidence-leaving rather than something a person watches ^[11].

The mechanism is more interesting than the philosophy. VMR keeps the entire request body as json.RawMessage and parses only the model and stream fields, so every other byte passes through untouched ^[17]. The reasoning is defensive: a proxy that does parse, mutate, stringify on each request puts unknown fields at theoretical risk of being reordered, escaped or dropped, and the author wants new upstream parameters and event types to work the day a provider ships them rather than the day VMR ships a release ^[18]. Cache affinity is the other half, pinning long conversations to one endpoint so routing does not invalidate the upstream prompt cache ^[5].

The numbers are self-measured and explicitly not third-party audited: 85 to 95 percent cache hit rates and roughly 50 to 70 percent cost reduction on cache discounts ^[6]. On an Apple M1 benchmark the author reports routing overhead of 0.3 to 0.9 ms at p50, 15MB RSS and about a 60ms cold start, out of roughly 31 thousand lines of Go with four direct dependencies in a 12 to 15MB static binary ^[7][3]. Resident memory therefore sits at the top of the shipped artifact's size range ^[20]. Even at the good end of the reported band, 5 to 15 percent of requests still miss ^[21], which is the honest framing: a distribution shift, not a guarantee.

The exclusions are policy. Every feature is tested against one question, capability or complexity, and dashboards, user management, billing, prompt management, plugin systems and MCP frameworks were all rejected at the door ^[15]. LiteLLM-class translation gateways were rejected because the protocol translation layer must be maintained forever as upstream event formats evolve ^[12]. Electron control planes such as Claude Code Router were called deep and well-made but oversized for the job ^[13]. Hardcoded URLs with manual switching offer no failover, no health probing and no audit trail ^[14]. Go was picked because the workload resembles a reverse proxy more than a desktop GUI ^[16], and because four module dependencies compare well against a 50-plus npm tree with Electron and React ^[19], which is at least a twelvefold gap on the author's own comparison ^[22].

Two things to watch. Byte fidelity is cheaply testable: if requests really do arrive upstream byte-identical to a direct connection ^[4], someone other than the maintainer can verify it, and the cache figures can be checked against a provider's own usage reporting. And the always-leaving-evidence requirement ^[11] sits in tension with rejecting every reporting surface ^[15]; the audit trail the author faults manual switching for lacking ^[14] has to live somewhere, and where it lands will decide whether the no-dashboard position survives its second year.