Data-center GPU on NVIDIA's Ada Lovelace architecture, used for AI inference and video workloads; powers AWS's g6.xlarge instance.
SageMaker costs 1.40x a plain EC2 instance per hour to serve one Gemma 4 vLLM build on the same T4 or L4 GPU, a benchmark on dev.to finds. With decode speed matched within 2%, the extra 40% goes to the managed layer around the GPU.
Reality
Gemma 4 decodes on SageMaker's smallest GPU, a T4, at 0.8x an L4's speed with matching outputs from a patched vLLM image, a dev.to benchmark reports. The T4 is the cheaper choice per token only when it rents for under 80% of the L4's hourly rate.
Reality
Repacking Gemma 4's QAT weights with 4-bit embedding tables made decode up to 1.39x faster on one SageMaker L4, according to a dev.to benchmark series. For teams serving Gemma 4 on vLLM, how the weights are stored becomes a setting to measure alongside model size.
Reality
FastGPU's September 27 snapshot of 28 GPU clouds puts the cheapest hyperscaler H100 at $5.38 an hour, 3.0x the $1.79 market floor. That premium buys IAM, managed services and credits, and it is worth paying when a team actually uses them.
Reality
Developer xbill9 rebuilt Google's QAT Gemma 4 26B as int4 and fit it on one TPU v6e with 53,888 tokens of KV cache, against 3,456 for RedHat's FP8 build. Throughput nearly doubles, with accuracy checked on one classification suite.
Reality
Gemma 4 E2B's 4-bit QAT checkpoint decodes 2.05x faster than bf16 on one SageMaker L4, according to a dev.to benchmark. The swap also frees 18% of GPU memory for a 20% larger KV cache, though it runs only through the vLLM container because JumpStart lists no QAT build.
Reality
A pre-registered run reads Gemma's label logits on one 24 GB GPU and scores it against the hosted API on the same 3,880 records, where it is level on yes/no questions and 4.5 points behind on multiple choice. One temperature fitted on 50 labels closes the calibration gap.
Reality
Identical code and identical weights diverge 3.7x on decode between a T4G and an L4, because Turing has no bf16 datapath and XLA emulates one in fp32 while the instance keeps returning 200s and green health checks.
Reality
NVIDIA documents MIG as compute-only, so headless Chromium inside an H100 slice opens no GPU context at all and quietly falls back to CPU rasterising, which surfaces as a frame rate problem rather than a missing driver.
Reality