You need to quantize a model for deployment. What actua...

The first thing that actually breaks, well before overall model quality collapses, is precision loss driven by outlier values in specific activation dimensions. In practice, a small number of channels in a layer’s activations carry much larger magnitude than the rest — and if you quantize the whole tensor uniformly, that handful of outliers forces the quantization range to stretch wide enough to represent them, which crushes the precision available for every other, normally-scaled value in the same tensor.

This is why naive uniform int8 quantization degrades quality faster than the “average” bit-reduction would suggest, and why the quantization methods that actually work well in practice (mixed-precision schemes, outlier-aware quantization) specifically detect and separately handle those outlier channels — keeping them at higher precision or isolating them from the rest of the tensor’s quantization range — rather than quantizing everything with one uniform scale.