HAProxy's 206ms token lag comes from an MSG_MORE tag on every in-flight body write

MSG_MORE is a flag on the send call. Nothing inside the proxy is collecting tokens; the flag tells the kernel that more data is coming and to hold what it was just handed ^[8]. HAProxy already sets TCP_NODELAY on its TCP sockets, so Nagle's algorithm is not the thing holding the token ^[7]. A response body that arrives in one piece clears the flag straight away. A token stream never does: 60 bytes every 50 milliseconds, with a round trip between each write, so the corked writes sit in the kernel until the tag clears or a timer cycle releases them ^[18].

The configuration manual describes the effect in the same units the benchmark measured. The system "waits for enough data to be available in order to only send full packets. Typical delays are around 200 ms per round trip" ^[10]. The measured 206 milliseconds is about four of the emitter's 50 millisecond intervals ^[23].

For that figure to transfer to your deployment, your upstream has to emit frames that are small relative to a packet and pause between them. The benchmark author says frames-per-read is a property of the stream shape; it moves when the stream does ^[14]. His 1.1 KB frames are roughly eighteen times the 60-byte payload ^[22], and the delay they produce is about a quarter of the 206 milliseconds ^[21].

Clearing the flag moves the cost onto the traffic HAProxy is usually bought for. The troubleshooting guide says Nagle "definitely remains enabled when forwarding an HTTP body (and this contributes to the performance improvement there by reducing the number of packets)" ^[16]. The reason is header overhead, described in RFC 896 in 1984: a sender holding a small amount of data waits for a full packet's worth, or for the receiver to acknowledge the previous packet ^[11]. A 60-byte write carries header overhead, and those headers can double the bytes transmitted ^[12]. For a download or a bulk API response, 40 milliseconds of accumulated batching once per connection is invisible to whoever is waiting on it ^[19].

The default was tuned around a specific idea of interactive traffic. Batching is skipped in pure TCP mode and in tunnels, and WebSockets and CONNECT requests are documented as unaffected ^[17]; the author says the picture at the time was a tunnel or an upgraded socket ^[26]. That list left out an HTTP response body emitted one token at a time.

The account comes from inside the vendor. "NOTE: I work at HAProxy, so I'm probably a bit biased," the author wrote, and said the post is his own opinion and research, not official from the company ^[5][24]. He says the behaviour is reversible with one line of config. The post does not name the directive ^[25].