NVIDIA Turing-architecture GPU (compute capability 7.5) paired with Arm-based CPUs in AWS G5g cloud instances for inference workloads.
Each rig had been quoting throughput from a field its own server invents, so the vLLM comparison had never actually run. Switching to wire-level timing cost under three dollars of instance time and produced five corrections.
Reality
A dated pricing run puts the Graviton2-hosted G5g 20 percent below the Intel G4dn per hour, but that host resolves a container image with no kernels for its own T4G, so it compiles vLLM before serving a token.
Reality
Identical code and identical weights diverge 3.7x on decode between a T4G and an L4, because Turing has no bf16 datapath and XLA emulates one in fp32 while the instance keeps returning 200s and green health checks.
Reality
The cheap fractional-GPU tiers top out at 3 GB of device memory, so the weights pick the instance and hand you a Graviton2 host with a 2018 Turing card. The interesting cost is the toolchain, not the hourly rate.
Reality
One hand-written JAX port runs on TPU v5e, v6e and a Turing T4G, but the quantized fast path stays on TPU and the compute dtype has to be read off the live device, because neither leak ever shows up as red in a log.
Reality