A free-tier exit probe gates CI on one latency sample out of 200

The percentile function decides more than it appears to. It sorts the latencies and picks index min(len-1, int(round(q * (len-1)))), with no interpolation ^[9]. At the default 200 requests ^[7], that puts p95 at index 189, the 190th smallest of 200 values, with ten observations sitting above it ^[1]. Ten slow calls can hide above the gate without moving it. If those ten failed, the error rate would be 5 percent, more than twice the 2 percent default budget, so the error gate trips before a timeout can shift the p95 estimate ^[3].

That error gate is coarse at this sample size. Two percent of 200 is four, so four failures still print pass and five, at 0.025, print exit ^[2]. Failures also land in the latency list, because it is built from every result and a call that raises returns its elapsed milliseconds alongside ok False ^[10]. A call that hangs to the 30-second timeout ^[7] therefore counts once in error_rate and again in max_ms.

The listing leaves out two of the four measurements the post asks for. one_call issues a single POST per invocation and returns a dict on exception; it never retries, and it never sits idle before a call ^[11]. Retry amplification and the cold-path penalty are the first two of the three effects the post blames for the curve ^[3], which leaves queue debt as the one the harness actually instruments, through wall_seconds ^[5]. The report already prints drain time ^[8].

The prompt is "Reply with exactly one word: pong" with max_tokens 4 ^[6]. That measures admission, queueing and a few tokens of decode. For the p95 to transfer to a nightly coding-agent batch, the batch would need output lengths in the same range as a four-token completion, and a connection pattern like the harness: one AsyncClient, max_connections set to the concurrency, all 200 calls sharing the pool ^[12]. Separate worker processes each make their own handshake. The default budget also passes an endpoint that needs six seconds to return four tokens ^[7], and at that ceiling 200 calls at concurrency 8 run as 25 waves for 150 seconds of wall time ^[4].

The free token allowance the post cites is the one listed in the terms in effect on 2026-09-15, and the post tells readers to check the current terms page because quotas and hardware change often ^[16][17]. The harness reads its credential from the environment variable FREE_TIER_KEY, and the call site still contains args.api_key if False else api_key even though the parser never defines --api-key ^[13]. The dead branch would raise AttributeError if it ever evaluated. The literal False means it never does.

Before gating a build on this I would raise --requests until p95 rests on more than one observation, or run the probe several times and compare reports. A retry wrapper around one_call, counting attempts per logical call, would cover the effect the post puts first ^[3]. A sleep past the provider's idle window followed by one timed call gets the cold-start number the post asks for ^[5]. For a nightly batch the threshold worth arguing over is wall_seconds against the length of the window, and wall_seconds is already in the report ^[8].