Unbounded Tokio tasks grew a 100 MiB Rust pod to 3.7 GiB during traffic bursts

The growth factor is roughly 38x. 3.7 GiB is about 3,789 MiB, set against a 100 MiB baseline ^[1]. The team describes the failure as a chain: bursts, then unbounded tasks, then memory accumulation, then pod memory exhaustion ^[15]. No step in that chain depends on a memory leak.

Removing garbage collection is what made the port pay off. According to the team, Rust eliminated GC pauses and made latency more predictable ^[17]. On the replication layer, steady state settled at 27 to 31 microseconds. A gap of about 1.5 microseconds between nodes also became visible, where GC noise had hidden it before ^[10]. Tokio's scheduler gave the services high concurrency at low CPU ^[16]. Pods settled near 0.22 cores, against 0.4 to 0.6 before the port ^[5]. Because accepting work costs almost nothing in CPU, the runtime also has no natural point at which it refuses a request, and every accepted request becomes a task holding memory.

The autoscaler episode comes from the same missing limit. Scaling out adds pods, but no pod limits its own intake, so each new replica can build its own queue of waiting tasks ^[8].

I think the ingest edge is the right place for the cap. A request refused there never becomes a task, so it never allocates the buffers and state a waiting task would hold ^[7]. The write-up does not say which primitive enforces the cap or how the limit was sized. Shedding also moves cost onto callers. Separately, the team replaced its custom TCP protocol with gRPC and a store-and-forward model to improve message durability and retries ^[14].

The latency figures are claims about this workload, and part of each gain is architecture. The load balancer improvement, about 5.9x, came with a proxy swap ^[3]. Presence API peak memory fell from 3.4 GiB to a 256 to 512 MiB band after a redesign ^[4]. The team's own comparison of two Rust implementations puts connection pooling and data layout on par with language choice ^[13]. The publish API went from about 350 to about 50 microseconds, roughly 7x ^[3][2]. For that gain to repeat elsewhere, a service would need a tail dominated by GC pauses and a budget for comparable redesign work.

The measurement is careful. Every dashboard in the post existed before the migration, so each before-and-after figure is compared against a recorded baseline ^[11]. The cost account is direct too. The port from Python, Go, JVM and C ran over three years ^[1]. Engineers fought the borrow checker for the first six months, and productivity improved by month eight ^[12].