Do the arithmetic per byte and the 203 GB case looks worse than merely large. Weights loading for the 64 GB model took about 29 seconds, roughly 2.2 GB/s [10][21]. The 203 GB model took about 423 seconds, roughly 0.48 GB/s [11][22]. Same instrumented path, about 4.6 times less throughput per byte on the bigger model [23], and that model was split across four GPUs under tensor parallelism, so there were more consumers waiting on the stream, not fewer [11]. The New Stack report puts the blame on the calling pattern rather than on link capacity [4]. If 2 percent of available bandwidth is in use, the lever is requests in flight, not a fatter pipe: at full utilisation those 423 seconds would arithmetically land near 8.5 [24]. Nothing reaches full utilisation in practice, but the ratio sizes the argument you are having.
The compilation phase behaves the way the authors say it does, tracking graph complexity rather than parameter count [12]. The 64 GB model spent about 53 seconds in torch.compile and the 203 GB model about 34 [10][11], so the smaller model compiled 56 percent longer [25]. Below roughly 100 GB compilation dominates; above it, transfer does [12]. The output is deterministic, identical on every start [3], and engine initialisation itself swings between 30 and 120 seconds depending on whether that compilation is cached [9], so the cache pays twice. A cache whose output is byte-identical every run being off by default is a decision somebody made once and nobody revisited [5]. The warm-path fix is described as environment variables plus a volume mount that work on any Kubernetes cluster [15], though the piece does not print them and defers the weights loading and compilation cache configuration to an AWS document [19].
What would have to be true for the numbers to transfer: p5.48xlarge instances under EKS Auto Mode, S3 traffic routed directly with the NAT Gateway out of the path, and container images in a private ECR repository in the same region as the compute [17]. The authors are explicit about which part travels. The 80 to 93 percent improvement ratio held on P-family and G-family instances, while absolute cold-node times vary with network bandwidth and CPU count [18]. A cross-region image pull changes the baseline you are improving against, not just the result.
They also define the metric they are optimising, which is more discipline than this category usually gets: time to first token served runs from pod creation to the first inference response leaving the GPU, and per-request TTFT starts where it ends [2]. Of the measured 480 seconds, about 120 are node provisioning and framework initialisation that no application-layer change removes, leaving roughly 360 addressable [1][14][26]. Karpenter's provisioning step alone is 60 to 90 seconds [7], so a sub-minute figure cannot describe a freshly booted node; it describes a pod restarting where node, driver, image and cache are already resident [27].
The 2026 primitives each cover a layer. OCI image volumes are stable for model delivery, Dynamic Resource Allocation replaces opaque GPU integer counts with structured attributes, and Gateway API has inference-aware routing extensions [20]. None owns the whole path, and the six layers, including an 8 to 12 GB compressed image pull, have different bottlenecks, different fixes and different owners [6][8]. If your platform team owns two of them and your model team owns the rest, eight minutes is nobody's bug.
