`docker pull vllm/vllm-openai` resolves the tag against the host architecture before anything CUDA-related happens, and the two platforms behind that one tag disagree about which GPUs they serve. The amd64 config blob (sha256:2286e8533ca8) lists `TORCH_CUDA_ARCH_LIST=7.5 8.0 8.6 8.9 9.0 10.0 12.0` and contains sm_75 [8]. The arm64 blob (sha256:2a7cde230b59) starts at 8.0 and has no sm_75, same tag, same day [9]. So a Graviton2 host resolves, on architecture alone, to the image with no kernels for the accelerator attached to it, and the Dockerfile appends no `+PTX`, which removes the JIT fallback as well [10]. That box builds vLLM from source before it answers a request [10]. Billing runs during the compile.
The GPU itself checks out fine. vLLM independently sized the KV cache at exactly 329,579 tokens on both instances [7], which is about as close to proof of an identical GPU-side configuration as a running system will give you. The author's reading is that the mismatch is a packaging decision by the vLLM project rather than a property of either CPU, and the largest single cost difference between the families [11].
Invert the discounts and you get the number a capacity plan actually spends. Twenty percent cheaper per hour on demand [4] means the Intel box costs 25 percent more per hour [20]. Fifty-nine percent cheaper on spot [4] means the Intel box costs roughly 2.44 times as much, a 144 percent premium [21]. That premium buys an image that already has kernels in it.
The L4 ladder does not close the gap either. Take the largest of the three cheaper slices as the 0.4750 tier: g6f.4xlarge doubles both sides, 11,444 MiB against 5,722 and 0.95 against 0.4750 [22]. Framebuffer per dollar-hour is flat, and the 11,444 MiB slice leaves about 1,675 MiB above a 9.54 GiB checkpoint for KV cache and everything else [23]. The 10,246,621,918 bytes in the mmap error is the same 9.54 GiB object, not a second one [24].
On the host side the split is per-vCPU RAM: 4 GiB on G4dn against 2 on G5g [12]. That is why g5g.xlarge lands at about 7.5 GiB usable and errno 12 comes back from the mapping rather than from residency; a swapfile fixes it, and g4dn.xlarge's 16 GiB maps the checkpoint with no swapfile configured at all [13][14].
For the cost-per-token ranking to transfer to your workload, several things have to hold. You have to pull the published tag rather than build an arm64 image with 7.5 in the arch list or `+PTX` enabled, because that is the difference being priced [11]. Your model has to sit in the same 9 to 10 GiB band, since a smaller checkpoint moves the mmap ceiling out of the picture [13]. And your instances have to be short-lived enough that a from-source build is a real line item; on a box you keep for a month, the compile amortises and the 20 percent hourly discount is the number that wins [15]. The supplied text names the comparison but does not reproduce the throughput rows, so the mechanism is checkable and the ranking is not [25].
Worth noting what the deployment does well. The Hugging Face token is read from Secrets Manager at boot instead of user data, because instance metadata is readable by anything on the box, and the fetch is wrapped in `set +x` because the bootstrap runs under `set -x` and bash traces assignments with their values [17]. Access is SSM only, with no inbound SSH rule and no key pair [16]. The MCP layer is one Python file over stdio with three dependencies [18].
