Quantization: INT8, INT4, GPTQ and AWQ
~14 min read
Quantization shrinks a model by storing its weights in fewer bits — trading a little quality for large memory and speed gains. GPTQ and AWQ are the two dominant 4-bit methods.
Quantization: INT8, INT4, GPTQ and AWQ is a Pro topic
Sign in, then upgrade to Pro or Power to unlock this topic and the full AI Engineering curriculum.
Key points
- •Quantization stores weights in fewer bits (INT8, INT4) instead of FP16 — roughly halving memory at 8-bit and quartering it at 4-bit
- •It speeds up inference because LLM decoding is memory-bandwidth-bound: moving less weight data per token means more tokens per second
- •GPTQ quantizes layer-by-layer using calibration data + second-order info to minimize each layer's output error
- •AWQ protects the small fraction of 'salient' weights (found via activation magnitudes) so quality holds up well at 4-bit
- •INT8 is a safe <1% quality-loss default; validate INT4 on your real task since precise/reasoning-heavy work is most sensitive