Skip to content

Build1 publisher2 min readPublished

One reused socket served 199 of the 200 GETs behind a 12 ms mean

Field notes from a two-day hunt for a missing 12 milliseconds end at the client, where a reused requests.Session let one cold request and 199 warm ones share a mean. The figure describes a long-lived worker with a warm pool.

The Engineer · Build desk

Illustration accompanying One reused socket served 199 of the 200 GETs behind a 12 ms mean

What happened

  • A laptop script reported twelve milliseconds for two hundred sequential GETs against a staging endpoint, and the mean was good enough to go into a design doc.
  • Production p95 for the same endpoint sat near a full network round trip, and nobody could reconcile the two graphs.
  • The author spent forty-eight hours treating the gap as a server problem before finding a client measurement bug.
  • Only the first request paid for the handshake and TLS; urllib3 pooled the connection and requests two through two hundred reused it, barely touching DNS.
  • Printing every sample instead of the mean showed sample zero running several times slower than sample one.

Compiled by The EngineerSomething wrong?How this is made

Why it matters

  • decision A mean is not sufficient evidence in a latency review any more; the per-sample list with sample zero visible is what separates a fast handler from a warm socket.
  • cost The bill for a smoothed mean was two days of server-side debugging on logs, middleware and a tracing exporter that were never in the path.
  • constraint A design doc holding the pooled figure specifies steady-state worker latency only, and capacity work for per-invocation callers needs its own measurement.

urllib3's pool is keyed by host, so the second call to the same host gets back the socket the first one left behind [5]. Run that loop against any remote HTTPS URL and the samples cluster tightly, and the field notes attribute the tightness to the pool rather than to a handler that got faster [17]. Averaged across 200 samples, the single request that paid DNS, TCP and TLS carries 0.5% of the mean's weight; in the reader-runnable version, with N set to 50, it carries 2% [2]. "Averaging cold and warm rows is not a measurement so much as a smoothing function with extra confidence," the author wrote [11].

The search went to the server first. Application logs, then framework middleware, then a tracing exporter that got disabled on suspicion of ten milliseconds, and none of it moved the production graph [8]. The process was restarted and the local resolver cache flushed, while the session stayed open between the timed calls [9].

"I was measuring keep-alive reuse, not the cost a cold cron job or a new pod actually pays," the author wrote [7]. The second harness splits the three cases that get casually mixed together: one long-lived Session for the whole loop, which is what a worker process usually does; a new Session per call, which is closer to requests.get() and to many CI checks; and the same pattern sending Connection: close, which forces the handshake back into view [13]. In the fresh-session path, take() builds the Session inside the loop and closes it in a finally block, so every sample there pays the full setup [15].

The second harness has its own limits. With N at 30, its p95 is sorted(xs)[int(0.95 * 29)], which is sorted(xs)[27]: the 28th of 30 sorted samples, third from the top, so two samples sit above the number it prints and a single outlier sets it [3][14]. Both harnesses call raise_for_status() inside the timed section, and the first one has no handler around it, so a non-2xx ends the run instead of showing up as a slow sample [4][12]. The post says the labels matter more than any absolute millisecond value from one laptop, and the text stops mid-sentence before the three rows' numbers [16].

What to watch

  • Whether the pooled, fresh-session and Connection: close rows get published with first, rest_p50 and p95 figures against the same endpoint.
  • What the production caller actually is: a long-lived worker with a warm pool, or a per-invocation process that handshakes every run.
  • Whether the design doc that carried the 12 ms mean is reissued with sample zero included.
Loading claim ledger
Loading source directory links
Loading share composer
Loading topic controls
Loading related stories