Quantizing value residuals and casting softmax to FP8 speeds Nunchux's attention kernel 1.46x on an H200

Take the H200 figure and push it through a denoising step. A 1.46x attention kernel means the call finishes in about 0.685 of the time BF16 FlashAttention-4 needed ^[2]. If attention is half the step, the step runs about 1.19x faster; if attention is 70 percent of the step, about 1.28x ^[16]. The authors argue that attention grows as a share of the workload as resolution and duration rise ^[10], so the top of that range belongs to long, high-resolution clips. Runtimewire printed the reported speedups under the heading "Kernel gains are larger than clip gains" ^[11].

V-Smooth works on the value side, where earlier low-bit attention methods mostly smoothed queries and keys before quantizing ^[5]. Per block, it clusters similar value tokens online, subtracts the block mean and quantizes the residual; the mean goes back in during the online softmax, so one pass over the values is enough ^[4]. The clustering happens inside the attention call. Its cost is already inside the 1.46x.

ExpCast-FP8 goes after the other end. The baseline path computes an FP32 exponential and then converts the result for what follows; ExpCast-FP8 replaces both with a fused multiply-add that maps log-domain scores straight into FP8 probability codes ^[6]. That collapses two operations into one instruction.

The largest numbers in the paper are 2.27x on an RTX PRO 6000 and 3.58x on an RTX 5090, and they are a different measurement: V-Smooth alone at four-bit precision, on workstation and consumer parts ^[3]. The eight-bit row is the one that applies to data-center parts ^[2].

VC-Attention works on a model as trained, so it goes in at the kernel level instead of being baked into each checkpoint during training, which runtimewire notes matters for a provider serving models from several labs ^[7]. The paper ran Wan2.2, LongCat-Video, HunyuanVideo-1.5 and MiniMax H3 over 100 MovieGen Bench prompts each, with prompts and random seeds shared across methods ^[8]. The material on hand reports kernel times, not clip times or quality scores.

Muyang Li and Jun-Yan Zhu framed the company around latency, cost and quality in a September 3rd launch post. "Video presses hardest on all three," they wrote ^[12]. MiniMax's September 18th post filed VC-Attention with FlashAttention and SageAttention under what it called the "dense, done faster" branch, the one that keeps the full attention pattern and makes each interaction cheaper ^[9]. Nunchux can stack that on Li's earlier sparse-attention work and cut both the number of interactions and the cost of the ones it keeps ^[13]. The company's chief system architect, Zhekai Zhang, led architecture and CUDA-kernel work on SpAtten and the Nunchaku diffusion inference engine ^[14].