Build1 distinct publisher3 min readPublished
Go's answer to unbounded fan-out was bounded concurrency plus cooperative cancellation, and both port to agent runtimes. But a slot count is the wrong unit when the scarce resource is tokens per minute.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
Read the corrected Go snippet in execution order. The goroutine is created first, and the semaphore acquire happens inside it, in a select against ctx.Done() [7]. So the cap of 500 governs concurrent process() calls, not goroutines: a request carrying 200 subtasks still creates 200 of them [3]. For cheap in-process work that is the right tradeoff, because a parked goroutine costs a few kilobytes and no scheduler time. It is still worth knowing which number you bounded.
The scale figure converts usefully. If the bursts carried 50 to 200 subtasks per request [4], a million goroutines works out to roughly 5,000 in-flight requests at the top of that range and 20,000 at the bottom [1]. The fan-out multiplier set that number, not the arrival rate. It is also one account, published on dev.to and originally on tamiz.pro [14], reporting that scheduler overhead became significant without giving a figure for it [6].
The Python remedy is where I would stop the review. asyncio.wait returns done and pending, and the snippet binds the second to an underscore [11][4]. The timeout does not cancel the pending tasks; the caller just stops waiting for them. The write-up's own diagnosis is that a goroutine leak becomes a token leak [10], and this is the line where that happens, because abandoned work is still billed work. The comprehension also reads r.result rather than calling it, so treat the sample as illustration rather than a drop-in [5].
Then the unit. A semaphore counts calls [11]. The scarce things named are rate limits and token budgets [10], and each of the 20 to 50 calls in a fan-out carries its own context window [9]. Ten slots bound calls in flight while tokens per minute stay free to spike, because a 200k-context call and a 2k-context call occupy the same slot [6]. For max_concurrent=10 to transfer to your service, two things have to hold: your subtask contexts are roughly the same size, and your provider quota is denominated in requests. Mine is denominated in tokens.
The cap also has a price the piece does not name. Fifty subtasks through ten slots is five waves, so plan wall-clock lands near five times a single call even when nothing fails [2]. That is the trade being made: a rate-limit error becomes a latency multiplier, paid by the user who asked for the plan.
Which is why I would import selectively. Cancellation propagation generalises as written, including the invariant that every node in the call tree inherits a context that can be cancelled and that the runtime enforces none of it [12]. Resource budgeting generalises only after you re-denominate it in the unit your invoice uses [2]. The 500 and the 10 belong to someone else's workload.
Ranked by verification strength, evidence, and original report placement.
The account labels the plan-and-execute loop that awaits execute_agent for each subtask in sequence a common anti-pattern in agent frameworks and unbounded fan-out.
Without concurrency limits, the account says you hit rate limits, blow your token budget and degrade response quality for all concurrent users, and the goroutine leak becomes a token leak and a latency cascade.
The lessons the account names from that scale are structured concurrency, cancellation propagation, resource budgeting and observable failure.
Without cancellation on ctx, every in-flight goroutine survived until its upstream request timed out or the process was killed.
The fix kept the goroutines and added boundaries: a buffered channel used as a semaphore with a comment of max 500 concurrent workers, acquired inside each goroutine via a select whose other case returns on ctx.Done().
A single user request can fan out to 20 to 50 parallel LLM calls, each with its own context window, API latency and error surface.
Distinct publishers with included, body-backed reporting in this cluster.
dev.to
1 article · August 27, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
build
Say "reliably" in a webhook spec and you have bought the whole distributed systems curriculum1 distinct publisher
build
SSE in Go breaks twice before your handler runs: an illegal header, then a 30-second timeout1 distinct publisher
build
Invoked in three runs, executed in none: the cost rule that never got asked1 distinct publisher
build
Your meter now runs on someone else's machine: signed receipts, fsync, and failing open1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Illustrative code, no measurements
Everything rests on one cross-posted blog entry. The Go and Python listings are verifiable as printed and support the pattern-level claims, but the load-bearing factual assertions — 1 million concurrent goroutines, significant scheduler overhead, accumulating memory, rate-limit and token-budget blowouts in agent systems — carry no metrics, no named system, no postmortem and no second source. Two of the printed remedies are also internally flawed, which further weakens them as evidence of practice.
No adoption signal
The cluster contains no release, deployment, benchmark, pricing, license or usage disclosure. No agent framework, repository or production system is named as having adopted or rejected the described pattern, so adoption cannot be scored without inventing facts.
Overstated relative to what is shown
The framing — a 1M-goroutine war story and an 'identical' solution that ports to agent runtimes — outruns the material. The scale claim is unverified, the degradation unquantified, and the two remedies offered do not fully deliver the properties claimed for them: the Go snippet caps in-flight work but not goroutine creation, and the Python snippet abandons pending tasks at timeout instead of cancelling them. The resource-budgeting lesson is also mis-specified, since a 10-slot semaphore bounds concurrent calls rather than the tokens-per-minute quota the article says gets blown, and the unstated cost is that a 50-call plan becomes five sequential waves.
Author-platform visibility, no commercial stake disclosed
The item is a cross-post of the author's own site onto a developer publishing platform, an arrangement that rewards reach and authority-building through a war-story frame ('1M concurrent threads', 'the new wave of AI engineering'). No vendor, product, sponsorship or funding relationship is disclosed or evident, and no tool is being sold, so the distortion pressure is reputational rather than commercial.
High confidence about the text, low about the world
The full body is available, so claims about what the article says and what its code does can be read directly and are assessed with high confidence, as are the arithmetic and code-reading derivations. Confidence about the underlying reality is low: one self-published, unverifiable anecdote with no metrics and no corroborating source, and no adoption evidence at all.