Build1 publisher2 min readPublished
One reused socket served 199 of the 200 GETs behind a 12 ms mean
Field notes from a two-day hunt for a missing 12 milliseconds end at the client, where a reused requests.Session let one cold request and 199 warm ones share a mean. The figure describes a long-lived worker with a warm pool.
The Engineer · Build desk

What happened
- A laptop script reported twelve milliseconds for two hundred sequential GETs against a staging endpoint, and the mean was good enough to go into a design doc.
- Production p95 for the same endpoint sat near a full network round trip, and nobody could reconcile the two graphs.
- The author spent forty-eight hours treating the gap as a server problem before finding a client measurement bug.
- Only the first request paid for the handshake and TLS; urllib3 pooled the connection and requests two through two hundred reused it, barely touching DNS.
- Printing every sample instead of the mean showed sample zero running several times slower than sample one.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- decision A mean is not sufficient evidence in a latency review any more; the per-sample list with sample zero visible is what separates a fast handler from a warm socket.
- cost The bill for a smoothed mean was two days of server-side debugging on logs, middleware and a tracing exporter that were never in the path.
- constraint A design doc holding the pooled figure specifies steady-state worker latency only, and capacity work for per-invocation callers needs its own measurement.
urllib3's pool is keyed by host, so the second call to the same host gets back the socket the first one left behind [5]. Run that loop against any remote HTTPS URL and the samples cluster tightly, and the field notes attribute the tightness to the pool rather than to a handler that got faster [17]. Averaged across 200 samples, the single request that paid DNS, TCP and TLS carries 0.5% of the mean's weight; in the reader-runnable version, with N set to 50, it carries 2% [2]. "Averaging cold and warm rows is not a measurement so much as a smoothing function with extra confidence," the author wrote [11].
The search went to the server first. Application logs, then framework middleware, then a tracing exporter that got disabled on suspicion of ten milliseconds, and none of it moved the production graph [8]. The process was restarted and the local resolver cache flushed, while the session stayed open between the timed calls [9].
"I was measuring keep-alive reuse, not the cost a cold cron job or a new pod actually pays," the author wrote [7]. The second harness splits the three cases that get casually mixed together: one long-lived Session for the whole loop, which is what a worker process usually does; a new Session per call, which is closer to requests.get() and to many CI checks; and the same pattern sending Connection: close, which forces the handshake back into view [13]. In the fresh-session path, take() builds the Session inside the loop and closes it in a finally block, so every sample there pays the full setup [15].
The second harness has its own limits. With N at 30, its p95 is sorted(xs)[int(0.95 * 29)], which is sorted(xs)[27]: the 28th of 30 sorted samples, third from the top, so two samples sit above the number it prints and a single outlier sets it [3][14]. Both harnesses call raise_for_status() inside the timed section, and the first one has no handler around it, so a non-2xx ends the run instead of showing up as a slow sample [4][12]. The post says the labels matter more than any absolute millisecond value from one laptop, and the text stops mid-sentence before the three rows' numbers [16].
What to watch
- Whether the pooled, fresh-session and Connection: close rows get published with first, rest_p50 and p95 figures against the same endpoint.
- What the production caller actually is: a long-lived worker with a warm pool, or a per-invocation process that handshakes every run.
- Whether the design doc that carried the 12 ms mean is reissued with sample zero included.