A 5h55m Azure OpenAI outage in Sweden Central needs a circuit breaker and a second route
Azure OpenAI and Foundry in Sweden Central failed for 5 hours 55 minutes on 29 September, according to a postmortem write-up summarised on dev.to. The post prescribes a per-call timeout, a circuit breaker and a second route, and we found two gaps in its sample breaker.
The Engineer · Build desk
Three figures. The sample circuit breaker's default threshold is 5 failures. Up to five failing calls each wait out a full timeout before the breaker opens. If half of all calls fail independently, the chance that five calls in a row all fail is about 3.1%, an illustrative assumption.
| Figure | Value | Note | Claim |
|---|---|---|---|
| Sample breaker's default failure threshold | 5 failures | 7 | |
| Failing calls that each wait out a full timeout, up to | 5 calls | before the breaker opens | 17 |
| Chance five calls in a row all fail, about | 3.1% | illustrative: half of calls fail | 15 |
What happened
- Customers saw intermittent request failures, higher latency and HTTP 5XX errors against both model and data plane APIs.
- The post argues that retries, built for blips, do harm over six hours: each request waits through several failed attempts, workers pile up and the recovering provider gets a flood.
- It sorts errors into three groups: a 429 is rescheduled, a client 4XX is fixed, and a timeout, connection reset or 5XX counts as an outage that needs a breaker and a fallback.
- The sample circuit breaker defaults to a threshold of five failures and a 30-second cool-down.
Why it matters
- decision Teams with an LLM call on the critical path have to choose their second route ahead of time: another region, another provider or a degraded mode.
- cost Until the breaker opens, up to five calls each wait out a full timeout, so the timeout value sets what the first users of an outage pay.
- constraint Thresholds tuned to this incident would rest on one author's summary. The post points readers to Azure's status history for the confirmed record and leaves the next-day networking incident unconfirmed.
The post frames the test as one question: "the question is what your code does during hour three." [13] A per-call timeout is the first answer. Without one, the post says, an LLM call can hang for minutes while holding a connection, a worker and a user [5]. It tells readers to set the limit from what a healthy call looks like: a few seconds for a short classification, longer for a long generation, and time to first token watched separately on streams [6].
The breaker is where we would change the code. Its `allows` check compares the clock with `openUntil` and nothing else [8]. A failure is recorded only after a call has rejected [10]. For a hung provider, that is when the timeout fires. So every request that arrives between cool-down expiry and the first recorded failure passes the gate [16]. The post says the next request acts as the probe [8]. Under steady traffic with a timeout of several seconds, that is many probes, not one. A half-open state that admits a single in-flight call would close the gap. The author calls the posted version enough for most services [14], and for a low-traffic app that holds.
The trip condition has a second limit. Because `success()` zeroes the failure count [9], the breaker opens only on five failures in a row [7]. The write-up describes the outage as intermittent. Suppose, for illustration, that half of all calls fail and that they fail independently. A given run of five calls then all fail 0.5^5 of the time, about 3.1% [15]. The summary gives no failure rate, so the 50% is our assumption, and the figure transfers only to a provider that fails at roughly that rate with no correlation between calls. A failure ratio over a sliding window would trip on that pattern. A consecutive counter mostly would not.
The fallback loop is the better piece of engineering. It skips any provider whose breaker is open and rethrows anything that is not an outage signal, so a malformed request never burns through the list [11]. When every entry is down it throws the last error, or `Every provider is unavailable` [11]. In our view that throw is where the degraded response belongs. The caller has to write it, because the loop cannot invent an answer.
What to watch
- Azure's official status history entry for 29 September, to check the 10:03 to 15:58 UTC window and the list of affected services against the write-up.
- Whether the next-day networking incident across several regions is confirmed, since a multi-region fault would not be covered by a second Azure region alone.
Clarity's read
What the record supports and how the coverage leans. The claims behind it follow.
Reality
- Evidence40
- Adoption
- Insufficient
- Hype gap−5
- Incentives
- Insufficient
- Confidence50
Claim ledger
Ranked by verification strength, evidence, and original report placement.
- [1]
The write-up mentions a separate networking incident the next day across several regions; the post's author draws no conclusions from it because the details have not been confirmed.
- [2]
The post's author says the official Azure status history is the place to look for the confirmed record, rather than anyone's summary.
- [3]
Retries are built for blips; against a six-hour incident every user request waits through several failed attempts, workers pile up, the queue grows, and the provider receives a flood when it recovers.
- [4]
The post sorts errors into three groups: a 429 means reschedule the work, a client 4XX means fix the request, and an outage signal (a timeout, a connection reset or a 5XX) needs a breaker and a fallback.
- [5]
Without a timeout, an LLM call can hang for minutes and hold a connection, a worker and a user.
- [6]
The post says to set a timeout on every call, chosen from what a healthy call looks like: a short classification should answer in a few seconds, a long generation needs a longer limit, and when streaming, time to first token is worth watching separately.
- [7]
The sample CircuitBreaker defaults to threshold = 5 and coolDownMs = 30000 (30 seconds).
- [8]
In the sample breaker, allows(now) returns now >= openUntil. The post says that after the cool down the next request acts as a probe; if it fails the count is still above the threshold so the breaker opens again straight away, and if it succeeds everything resets.
- [9]
In the sample breaker, success() sets failures to 0 and openUntil to 0.
- [10]
callProvider applies AbortSignal.timeout(provider.timeoutMs) to each call, and askWithFallback calls breaker.failure() only inside the catch block, after the call has thrown an outage error.
- [11]
askWithFallback loops over an ordered provider list, skips any provider whose breaker does not allow a call, rethrows errors that are not outage signals, counts outage errors against the breaker and moves on, and when the list is exhausted throws the last error or a new Error('Every provider is unavailable').
- [12]
The post advises strict timeouts, a circuit breaker for a provider that is clearly down, and a second route kept ready, whether another region, another provider or an honest degraded mode, because retries alone will not save you from an outage that lasts six hours.
- [13]
The post says: "the question is what your code does during hour three."
- [14]
Of the sample breaker, the post says: "That is the whole idea, and it is enough for most services."
- [15]
If half of all calls fail independently, the chance that five consecutive calls all fail is about 3.1%. The 50% rate is an illustrative assumption, not a figure from the source.
- [16]
As written, the breaker gate checks only the clock and a failure is recorded only after a call rejects, so every request arriving between cool-down expiry and the first recorded failure is allowed through, not just one probe.
- [17]
Before the breaker opens, up to five failing calls each wait out a full timeout.
- [18]
According to a postmortem write-up by Artur Markus, Azure OpenAI Service, Foundry Agent Service, Foundry Models and Cognitive Services in the Sweden Central region failed for 5 hours and 55 minutes on 29 September 2026, from 10:03 to 15:58 UTC.
- [19]
Customers saw intermittent request failures, higher latency and HTTP 5XX errors against model and data plane APIs.
Get this ledger inside ChatGPT. Ask about any of our stories and the answer brings its figure, each claim’s verdict and the sources. Free, no account.
Sources
1 independent publisher whose own reporting we read for this story.
- dev.toAzure OpenAI Outage: Keep Your AI App Running
1 article · October 15, 2026
Topics and entities
Follow any of these and your For You feed starts watching them — no settings page required.
Topics
- LLM reliabilityFollow
- Circuit Breaker PatternFollow
- Cloud OutagesFollow
Entities
- Azure OpenAI ServiceFollow
- Microsoft FoundryFollow
- Microsoft AzureFollow
- dev.toFollow
- Artur MarkusFollow