A Spend Firewall That Runs Before the API Call, and a Trace That Explains It Afterwards

Two open-source projects, AgentShield and Agent-Devtools, have shipped an integration that evaluates every agent transaction against spend rules before the API call goes out, then renders that decision inside the run's execution timeline ^[8][12][14]. The occasion, per a dev.to post by AgentShield's author co-written with Jacopo of Agent-Devtools, was a 3:07 AM agent run: 21 calls to an unnamed premium LLM endpoint at $133 each, 60 seconds, $2,800 ^[1][2].

The numbers are self-reported and the product of the same people selling the fix, so treat them as illustrative. They are also internally slightly loose: 21 times $133 is $2,793, so at least one figure is rounded ^[3]. The endpoint is not named ^[2].

What survives the caveats is the control-loop argument. The budget alert email fired four minutes after the spend began, which is roughly three minutes after the money was gone and four times the duration of the incident itself ^[4][5]. An alert is a control only if a human can act inside the failure window, and a 60-second window at 3 AM is not that. Manual monitoring fails in the same way, and the post is blunt that it fails specifically on weekends, holidays, and the night you forget ^[7].

Rate limits are the interesting case, because they feel like a spend control and are not one. Capping speed caps the rate of burn, not the total: at $133 per call, a limiter that allows one call per minute still delivers $2,793 in 21 minutes instead of 60 seconds ^[6][2][9]. You get a slower fire.

AgentShield's answer is a pre-execution gate: pure Python stdlib, zero dependencies, seven composable rules, and a kill switch, returning APPROVED, BLOCKED, or FLAGGED in priority order, deterministically, in under 1ms per evaluation ^[8][10][11]. On the reported incident, 21 evaluations at that latency add under 21ms to a run that spent $2,793 ^[13]. A single rule, transaction_limit with max_amount of $100, would have stopped it ^[15].

The rule list includes cascade cost estimates alongside transaction limits, daily totals, velocity, merchant allowlists, category blocks, and session budgets ^[10]. That is the one to be skeptical about: pre-execution control requires guessing the cost of a call chain before it runs, and the post names the rule without describing the estimator ^[10]. Blocking also converts a cost failure into an availability failure, which is a trade, not a free win.

Hence the second half. Agent-Devtools is a local-first causal debugger with visual replay, behavior diffing, and visibility into prompts, context, memory, retrieval, and tool calls ^[16]. The joining mechanism is deliberately unglamorous: AgentShield emits one agentshield.spend.evaluation event per evaluation as NDJSON to stdout or a file, or via an in-process callback, and its trace_id joins directly to the debugger's native run_id ^[17][18]. Every rule reports triggered, passed, skipped, or not_reached, so near-misses are visible and not just the rule that fired ^[19]. Amounts cross the wire as Decimal-safe strings rather than floats ^[20]. The emitter was shipped additively, leaving existing evaluate() behavior unchanged ^[21], and both sides ran independent end-to-end verification of each other's code ^[22]. The collaboration started from a comment on a LangChain issue about runaway agent loops ^[23].

Watch whether the cascade estimator holds up on real tool chains, and whether anyone publishes a blocked-call postmortem where the trace actually named the cause: a retry loop in a library, a poisoned tool response, or a bad prompt ^[24]. Without that, the firewall becomes whack-a-mole and the agent runs again tomorrow ^[24].