standard
NVIDIA 4-bit floating-point format used on Blackwell-generation hardware; the write-up reports QUASAR cutting held-out KL by roughly 30% in this format with no inference-path changes.
NVIDIA says Alibaba's largest open-weight model serves over 4K tokens/sec/GPU and 350 tokens/sec/user in FP8 on a GB300 NVL72. That figure is the self-hosting floor, not a benchmark.
Reality
NVIDIA's Nemotron 3.5 Lightning is now deployable from SageMaker JumpStart and fits on one GPU. The argument underneath it is about billing, not benchmarks.
Reality
A 66 GB checkpoint becomes 22 GB and, NVIDIA says, up to 4x faster throughput, but only after distillation teaches the quantized student to live with its own noise.
Reality
An August 2026 paper argues low-bit quantization-aware training converges high because its reconstruction step ignores which weights matter. The fix is reported to cost 1.4% of step time.
Reality