Inference Benchmarking

A twelvefold longer prompt costs this RTX 3090 only 12 percent of its decode throughput

nano-vLLM's decode cost fits in one expression, W/B plus KV(Tavg), and a batch-1 context sweep on Qwen3-0.6B shows why those two terms move in opposite directions as you add requests or lengthen prompts.