Loading today’s stories
Build3 distinct publishers3 min readUpdated
The August 17 postmortem describes a sidecar that hit its concurrency limit while its autoscaler measured the host, then a VS Code retry bug that kept Copilot down hours after everything else returned.
The Engineer · Build desk
Compiled by The EngineerSomething wrong?How this is made
GitHub published the root cause analysis for its August 17 outage on August 20, and the mechanism matters more than the 7 hours and 47 minutes it cost [2][1]. Two unrelated defects, one in an autoscaling policy and one in a VS Code retry path, degraded code hosting, authentication, CI and AI assistance in the same window [1][3][8]. The first defect is a measurement error. GitHub's analysis traces the start to an Istio service-mesh sidecar that reached its concurrency limit while its autoscaling policy monitored the host service rather than the sidecar's own capacity, so new resources never arrived as traffic climbed [3]. The saturated object and the observed object were different objects. Pressure then moved outward until four HAProxy nodes exhausted their flow limits and the gateway authentication path developed severe latency and failures [4]. Everything downstream of auth followed: GitHub measured peak error rates of about 20% for web and API traffic and roughly 50% for archive and raw-content downloads [5], with SAML and OIDC authentication, SCIM provisioning and Team Sync also affected [6]. The second defect set the headline duration. Slow responses from one internal endpoint activated a latent retry bug in VS Code, lifting Copilot Token Service traffic from a normal 7,000 to 9,000 requests per second to between 70,000 and 100,000 [8], an increase of roughly eight to fourteen times [23]. GitHub blocked token requests with HTTP 403 responses, reduced gateway retries and restored traffic gradually [9], and said scraping attacks against code-download endpoints complicated recovery [10]. Most services were back at 16:36 UTC, Actions stayed degraded until about 18:03 UTC, and the Copilot Token Service did not fully recover until 21:02 UTC [7]. Working backward from the published 7:47 duration, the incident began near 13:15 UTC, core recovery took about 3 hours 21 minutes, and the Copilot retry tail accounts for roughly 57% of the outage window [24]. A client-side retry loop, not the mesh, owns most of the number in the headline. GitHub says neither August incident was caused by a code or configuration change and that both were capacity failures [11]. The August 6 Actions incident ran more than nine hours, with 71% of workflow runs hitting infrastructure failures at peak and 75% of the remainder delayed by more than five minutes [12]. July's availability report counted eight service-degrading incidents [13], and in March GitHub had already acknowledged that rapid load growth, tightly coupled infrastructure and weak defenses against misbehaving clients were letting local problems spread [14]. The load is real: monthly commits went from 1.4 billion in April to 2.9 billion in August [15], roughly doubling [26], alongside about 130 million merged pull requests and 24 million new repositories a month [16]. The spending is real too, and did not prevent this. GitHub added more than 3 million CPU cores, 120 petabytes of high-speed storage and installed as much hardware as available power in its existing data centers allowed [17]. Azure now carries roughly 58% of platform load and half of all Git operations, up from 12% of platform load in May [18], a 46-point shift in three months [25], with GitHub aiming to move GitHub.com production traffic out of its own data centers by the end of 2026 and add a second Azure region [19]. Two committed changes are worth tracking: consistent retry limits, retry budgets and variable timeouts across service-to-service calls, and a review of lower-priority CPU and memory alerts for components that could fail during spikes [20]. The structural one is slower, namely isolating critical systems and removing shared dependencies between them [21]. Until that lands, treat GitHub auth, Actions and Copilot as one failure domain rather than three, and check what your own clients do when a token endpoint gets slow. Fedorov's post was his third public reliability update since March [27].
Follow any of these and your For You feed starts watching them — no settings page required.
Ranked by verification strength, evidence, and original report placement.
On August 17 GitHub experienced an outage lasting 7 hours and 47 minutes that disrupted github.com, authentication, GitHub Actions, APIs, pull requests, issues and Copilot.
In a postmortem published August 20th, GitHub CTO Vlad Fedorov said traffic reached a new peak while a critical infrastructure component in the Central US data center failed to scale with it.
Monthly commits on GitHub grew from 1.4 billion in April to 2.9 billion by August.
GitHub added more than 3 million CPU cores, 120 petabytes of high-speed storage and significant network capacity, installing as much hardware as available power allowed in its existing data centers.
Azure now serves roughly 58% of GitHub's platform load and half of all Git operations, up from 12% of platform load in May.
GitHub's root cause analysis traces the initial failure to an Istio service-mesh sidecar that reached its concurrency limit; its autoscaling policy monitored the host service rather than the sidecar's capacity, so new resources did not arrive as traffic increased.
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
First-party postmortem with component-level detail, corroborated by two independent write-ups
The core account is a named-executive postmortem from the operator of the affected system, and it carries falsifiable specifics: duration, per-service recovery timestamps, measured error rates, request-per-second baselines and peaks, and named components (Istio sidecar, HAProxy nodes, VS Code retry bug). Two outlets reproduce the same figures without contradiction, and the trade reconstruction adds details GitHub's own summary omits. The residual limit is that everything traces to a single originating source: no third party independently measured the error rates, and the RCA document itself is only referenced.
Platform-scale usage and a migration already past the halfway mark
This is not a proposal awaiting uptake: the affected system carries 2.9 billion commits, ~130 million merged pull requests and ~24 million new repositories a month, and the failure simultaneously halted auth, CI and AI assistance for organizations worldwide. The infrastructure response is likewise in production, with Azure already serving roughly 58% of platform load and half of Git operations against 12% in May. The remediation program itself is only partly landed - retry budgets, alert reviews and system isolation are described as in progress - which keeps this below the ceiling.
Mildly overstated progress narrative against a still-worsening incident record
The factual reporting is well calibrated - if anything the vendor post understates mechanics by omitting the failing components. The overstatement sits in the improvement framing: GitHub says it has made progress on reliability, yet eight degrading incidents in July, a nine-hour Actions failure on August 6 and a near-eight-hour multi-service outage on August 17 all postdate the March and April commitments, and the March diagnosis of tightly coupled infrastructure and weak client defenses describes exactly what happened. The New Stack's competitive framing also runs ahead of its evidence, naming Entire and Cursor's Origin as beneficiaries without any migration or usage data. Both are modest gaps, not inversion.
Vendor trust-repair disclosure plus a trade outlet with a competitive angle
The originating document is a reputational-repair post by the company that caused the outage, published under its CTO's byline and closing with a Universe conference registration promotion; its selection choices - impressive capacity totals foregrounded, failing components unnamed, prior incident counts absent - align with that interest. Microsoft's Azure is both the remediation and the corporate parent's platform, so the migration framing is not neutral. The New Stack pairs its coverage with named alternatives, and the trade reconstruction is comparatively disinterested but wholly dependent on the vendor's own numbers.
High on mechanics and scale, lower on whether remediation holds
The incident facts are consistent across three sources with no contradictions, and the quantitative disclosures are specific enough to constrain interpretation. Confidence is reduced by single-origin sourcing for every number, one derived timeline figure that depends on inferring a ~13:15 UTC start from the published duration, and the open question of whether retry budgets, alert reviews and system isolation will contain the next traffic peak during an unfinished migration.
invest
Cursor ships Origin to paying users as GitHub's outage count reaches 2571 distinct publisher
build
AI-written code fails the same four ways, and every gate you own reports green1 distinct publisher
build
Claude Code now outruns Copilot roughly two to one in JetBrains' survey of 15,000 developers1 distinct publisher
build
The defect tax on in-editor models is a review capacity problem, not a tooling one1 distinct publisher
Distinct publishers with included, body-backed reporting in this cluster.
1 article · August 20, 2026
1 article · August 20, 2026
1 article · August 20, 2026