Build1 publisher3 min readPublished
A 429 retry that swaps base_url sends production prompts to the drafting host
A dev.to checklist treats the model endpoint as a promotion surface with six gates, each requiring an artifact. Its enforcement is a grep over deploy manifests, so a base URL injected at runtime slips past.
The Engineer · Build desk
What happened
- A dev.to checklist sets four conditions before an AI feature leaves the branch: an allowlisted production base URL, an explicitly pinned model id, timeouts and retry counts set in prod config, and deny-by-default fallback.
- Any production manifest still containing localhost, 127.0.0.1, ngrok, trycloudflare, or a hostname tagged lab, dev, sandbox or draft fails the base-URL gate closed.
- Six gates each carry a pass condition, a fail-closed condition and a named receipt, with the base-URL receipt being an allowlist commit SHA paired with the secret-store version id.
- When a team cannot produce evidence for all four conditions, the post's instruction is to ship the feature with its flag off.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- constraint A text scan of Helm, Terraform and Compose overlays catches a hardcoded lab hostname and cannot see a base URL that arrives from the secret store at runtime, so enforcing that gate requires something that reads the store.
- decision Teams running one binary for drafting and production have to split its client defaults, because the labeling gate fails closed when both sides share the same defaults.
- exposure Deny-by-default fallback pushes the cost of a vendor quota error onto users, who see a failure for the duration instead of a quietly rerouted answer.
- cost Pinning the model id makes every vendor upgrade an explicit change with a named rotation owner and a runbook edit, paid by whoever tracks the vendor's deprecation cadence.
A retry wrapper is usually the least-reviewed object in the call path. On a 429 it sleeps and calls again. The dev.to checklist names the variant that hurts: a wrapper that treats HTTP 429 from production as a reason to call the drafting host [6]. A retry resends the request it already built, so the second host receives the same prompt and whatever production data went into assembling it. The post's rule: retries do not change the destination, and a retry that swaps `base_url` fails the gate, as do unlimited retries and exponential backoff with no cap [8].
That failure is invisible to the test most teams already have. The post says none of it shows up in a unit test that mocks `complete()` [7]. The receipt it asks for instead is a failing integration test that stubs production errors and asserts the lab host is never dialed [9]. The assertion lives at the transport layer, on which host got connected to, not on the function you stubbed.
The policy checker is a grep. CI scans deployable config (Helm, Terraform, the Docker Compose prod overlay, sealed secrets templates) and fails on denylisted hosts, and the post flags the half-measure of scanning only `src/` while ignoring `deploy/` [10]. Grep matches literal text. A base URL injected from a secret store at deploy time is in neither tree, and the base-URL gate handles that separately: the value in the production secret store has to match a committed allowlist, be HTTPS, carry no path wildcards, and not be a personal tunnel, with the receipt being the commit SHA of `inference-allowlist.json` plus the secret-store version id [11].
Four literal hosts and four tags make up the denylist: `localhost`, `127.0.0.1`, `ngrok` and `trycloudflare`, plus any hostname tagged lab, dev, sandbox or draft [5][19]. That catches the tunnel a colleague pasted into a helper. A vendor's own sandbox origin looks like every other vendor hostname, so only the allowlist constrains where prod can dial.
Whether the outage-to-leak framing transfers depends on two conditions holding in your shop. The fallback destination has to sit outside your trust boundary, and the retried request has to carry production data [20]. If the drafting host is a gateway you run inside the same VPC, a 429 fallback is a routing and billing surprise, and the payload stays where it was. The post cites no incident. It carries no measurements either, and it describes itself as "a copy-paste checklist plus a small policy checker you can run in CI" [15][21].
The part I would keep regardless of the leak argument is the receipt format. A commit SHA plus a secret-store version id makes the gate auditable after the fact, and it expires on its own: the next rotation changes the version id and the old evidence stops matching. A sign-off in Slack, the post says, does not count as a receipt [17].
What to watch
- A published incident report tying a 429 fallback to egress would move this from checklist to evidence.
- Whether vendor-generated clients stop baking a base URL into a default, which is one of the three failure modes the post names.
- Whether CI gains a check that reads the secret store itself, since that is where the base URL actually lives at deploy time.