other

SmoothQuant

A quantization technique that divides large activation channels by per-channel factors and multiplies the matching weight channels by the same factors, leaving the product unchanged while making activations easier to represent in 8 bits.

Current clusters

build1 publisher

A lone 50.0 activation coarsens the INT8 grid 62-fold for every other value in the tensor

Absmax INT8 takes one scale from the largest magnitude in a tensor, so a single outlier sets the step size for everything else. The LLM.int8() authors found those outliers sitting in about six feature dimensions.

Publishers:dev.to

Reality

Evidence62
Adoption: Insufficient
Hype gap+10
Incentives30
Confidence58