A 50-run repro loop finds a one-in-100 flake about two times in five

Fifty runs is a sample. Take the post's own cutoff, a test that fails once per hundred runs ^[12]. Assume the runs are independent and the chance of passing all fifty is 0.99 to the fiftieth power, about 0.605, so the loop surfaces a failure roughly two times in five ^[1]. Reaching 95 percent confidence takes 299 runs: 0.99 to the 299th is just under 0.05 ^[2]. Independence is a generous assumption too. The runbook tells you to repeat the job on several identical CI nodes because a flake can be node-specific ^[9], and fifty green runs on a healthy node still leave the test suspect.

"Reproducing flakiness is 80% instrumentation and 20% elbow grease," the author wrote ^[13]. The instrumentation is what makes the second step possible: without the container image and node label captured at failure time, the re-run happens somewhere else ^[7][8]. Re-running a pipeline is the cheapest diagnostic available, and it discards the evidence; the post lists repeated pipeline re-runs among the symptoms it is trying to kill ^[6].

Isolation here works by substitution. Replace downstream services with WireMock stubs and databases with Testcontainers instances, and if the failures stop, the nondeterminism was in the dependency ^[10]. Push the other way with stress-ng, tc network shaping or parallel test workers, and code whose timing assumptions hold on an idle runner begins to fail ^[11]. Awaitility is recommended on the same logic: a fixed sleep encodes a guess about how fast CI will schedule the next step, and polling with a deadline drops the guess ^[14]. For concurrency failures the protocol wants thread dumps, heap dumps and stack traces off the failing run ^[20].

Two of the five root causes borrow authority from outside work. The post says research on flaky tests identifies concurrency as a leading cause, and credits the Google testing team with the finding that larger tests are far more likely to flake ^[4][5]. It does not reproduce the figures or name the studies. For that size correlation to hold in your suite, your large tests would need comparable resource contention and comparable variance between runners. A team whose integration tests bring up three containers on a dedicated node has a different distribution than the one Google measured.

The ordering of the fix patterns is where the judgement sits: deterministic data, then timeouts, then mocks, then retries ^[15]. Retries come last because they are the only entry on the list that leaves the defect in the code. A disposable database per test, or a schema per test, removes the shared mutable state instead of tolerating it ^[17].