Build1 publisher2 min readPublished
Thread-per-connection turns a slow-drip flood into a kernel scheduling bill
A dev.to guide argues that two services under an identical connection flood diverge on CPU because one blocks a thread per connection while the other multiplexes thousands of them through epoll on a small fixed thread pool.
The Engineer · Build desk
What happened
- A dev.to guide argues that a botnet-driven connection flood can quietly double or triple a container CPU bill before anyone notices, depending entirely on how the listener is built.
- In the thread-per-connection model, once thread count reaches the thousands the kernel spends a growing share of every cycle choosing which thread runs next, and that overhead scales with connection count rather than traffic volume.
- An event loop juggling 100 connections and one juggling 100,000 run on the same small fixed set of OS threads, so the context-switching cost that grows with connection count does not arise.
Compiled by The EngineerSomething wrong?How this is made
Why it matters
- cost CPU footprint under load drives autoscaling behaviour and bin-packing density, and with them infrastructure spend, so a flood lands on the operator as a bill during an event nobody can schedule around.
- constraint Because synchronous CPU use rises non-linearly with connection count, headroom measured at normal concurrency does not predict headroom once connections multiply.
- decision If listener architecture sets the divergence, the CPU curve under a flood is settled when the framework is picked, long before the traffic arrives.
- capability On an event loop, CPU spend follows real work, so capacity can be planned against request volume even while connection counts are hostile.
A blocked thread waiting on a connection with no data still holds a kernel scheduling slot and its full stack allocation, according to the dev.to guide [5]. A flood of mostly idle, slow-drip connections therefore sits close to worst case for the blocking model, since it maximises thread count while minimising useful throughput per thread [8].
Count schedulable threads instead of connections. Ten thousand concurrent connections on a thread-per-connection listener is ten thousand threads, because each connection gets its own thread and blocks on I/O until it has data to process [3]. The same ten thousand on an event loop configured one thread per core, which the guide gives as the usual setup, is four threads on a four-core container [10]. That works out to 2,500 schedulable threads for every one the loop presents to the kernel [1].
The doubling-to-tripling figure is the guide's own, with no kernel version, thread-pool ceiling or packet rate published behind it [16]. For a number like that to transfer to your service, your runtime has to be able to reach thousands of live threads, because a bounded pool queues connections instead of scheduling them. Your attacker has to hold connections open instead of completing requests. And your per-request application work has to be small enough that scheduling overhead is visible against it.
Many synchronous frameworks serialise shared state behind a lock, including connection pools, logging buffers and rate-limit counters, and contention on that lock rises with thread count, so CPU that should be doing I/O work goes into spinning or waiting instead [6]. A rate limiter is a shared counter. On a blocking listener under a connection flood, every thread the flood created is contending for it.
The test works without the guide's numbers. During an incident, plot CPU against connection count and against request throughput; if CPU tracks connection count more closely than throughput, the guide attributes the cost to kernel scheduling and context-switch overhead and not to application logic working harder [9].
What to watch
- A published harness, with those numbers attached, would make the doubling-to-tripling range checkable against a real service.
- Incident write-ups that plot CPU against connection count and request throughput separately would show how often the guide's diagnostic actually fires.
- Whether managed runtimes and serverless containers expose the thread-pool ceiling that decides which side of this model an operator is on.