The model swap that cost three days: write the response contract before you pick a provider

The gateway is a for loop around an httpx call ^[6], and the thing it changes is a count. One function now knows what a provider response looks like, instead of every template that renders one ^[3]. That is the whole mechanism, and it is worth the hour it takes.

It is not provider independence, which is what the framing implies. The gateway reads its answer out of `choices[0]["message"]["content"]` ^[7], the same access path that failed in the first place ^[3]. Point it at something that returns a tool-call object and the parse raises, the exception is caught, and the loop advances to the fallback ^[8]. The bug has not gone anywhere. It now sits in one file, where one diff fixes it. Worth knowing which of the two you bought.

The suite is where the argument gets shaky. `test_response_shape_is_stable` asserts that the body's key set equals exactly `{content, provider, degraded}` ^[9]. Exact equality freezes the contract against addition: the first engineer who adds a `usage` or `model` field for cost attribution gets a red build for a change no consumer would notice. A product contract wants required keys and forbidden keys, not a fixed set.

Then the timing. `test_round_trip_stays_inside_budget` fails above 15 seconds ^[10]. The gateway allows each provider a 10 second timeout and lists two of them ^[8], so a hung primary followed by a slow but successful fallback can spend 20 seconds upstream while doing precisely what it was built to do ^[13]. The functional test and the budget test disagree about what correct means, and whoever meets that in CI will file the suite as flaky and stop reading it.

`degraded` compounds it. The flag is set by comparing the provider name to the string `fallback` ^[11], so it reports which upstream answered, not whether the answer got worse. As a routing signal that is fine. Called degraded, it invites downstream code to treat provenance as quality.

One reading note: the Python in the published piece has lost its indentation ^[16], so whether the 503 raise sits inside the `except` clause, ending the loop on the first failure, or after the loop cannot be read off the page ^[8]. If it is inside, there is no fallback at all, and the truncated degraded-mode test would be the only thing standing between that and production ^[18].

Provenance matters here because the arithmetic is the pitch. The article discloses MonkeyCode product outreach ^[12] and points at that project's free model access and free server as the cheapest place to run the exercise ^[5], with the suite in CI on every pull request and the free token allowance as the budget ^[17]. One hour spent against three days saved ^[15] works out to 24 working hours at eight hours a day, a 24x return ^[14], from one team of unstated size, reported secondhand. Contract-first survives the disclosure. The 24x does not have to.