# Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization

arXiv: https://arxiv.org/abs/2504.09629 · HTML: https://mlsanity.com/paper/2504.09629/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 63 of its 76 references that are in the map.
- First posted on arXiv 13 Apr 2025.

## Builds on (most cited first, 30 of 63)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration](https://mlsanity.com/paper/2306.00978/) (2023)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs](https://mlsanity.com/paper/2404.00456/) (2024)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers](https://mlsanity.com/paper/2206.01861/) (2022)
- [A White Paper on Neural Network Quantization](https://mlsanity.com/paper/2106.08295/) (2021)
- [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models](https://mlsanity.com/paper/2308.13137/) (2023)
- [A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Stories](https://mlsanity.com/paper/1604.01696/) (2016)
- [QuIP: 2-Bit Quantization of Large Language Models With Guarantees](https://mlsanity.com/paper/2307.13304/) (2023)
- [QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks](https://mlsanity.com/paper/2402.04396/) (2024)
- [SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression](https://mlsanity.com/paper/2306.03078/) (2023)
- [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019)
- [LLM-QAT: Data-Free Quantization Aware Training for Large Language Models](https://mlsanity.com/paper/2305.17888/) (2023)

## Built on by (most cited first, 30 of 41)

- [LPCD: Unified Framework from Layer-Wise to Submodule Quantization](https://mlsanity.com/paper/2512.01546/) (2025)
- [CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantization](https://mlsanity.com/paper/2602.05902/) (2026)
- [LoaQ: Layer-wise Output Approximation Quantization](https://mlsanity.com/paper/2509.06297/) (2025)
- [High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimator](https://mlsanity.com/paper/2510.10693/) (2025)
- [More Than Bits: Multi-Envelope Double Binary Factorization for Extreme Quantization](https://mlsanity.com/paper/2512.24545/) (2025)
- [OneComp: One-Line Revolution for Generative AI Model Compression](https://mlsanity.com/paper/2603.28845/) (2026)
- [KronQ: LLM Quantization via Kronecker-Factored Hessian](https://mlsanity.com/paper/2607.07964/) (2026)
- [D2Quant: Accurate Low-bit Post-Training Weight Quantization for LLMs](https://mlsanity.com/paper/2602.02546/) (2026)
- [Quantizing Time-Series Models As Dynamical Systems: Trajectory-Based Quantization Sensitivity Score](https://mlsanity.com/paper/2606.13300/) (2026)
- [Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMs](https://mlsanity.com/paper/2608.08506/) (2026)
- [NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models](https://mlsanity.com/paper/2602.06694/) (2026)
- [OJBKQ: Objective-Joint Babai-Klein Quantization](https://mlsanity.com/paper/2602.08376/) (2026)
- [TTQ: Activation-Aware Test-Time Quantization to Accelerate LLM Inference On The Fly](https://mlsanity.com/paper/2603.19296/) (2026)
- [Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimate](https://mlsanity.com/paper/2604.13806/) (2026)
- [OffQ: Taming Structured Outliers in LLM Quantization by Offsetting](https://mlsanity.com/paper/2606.07116/) (2026)
- [Why Does Post-Training Quantization Work?](https://mlsanity.com/paper/2609.11716/) (2026)
- [Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinement](https://mlsanity.com/paper/2609.25916/) (2026)
- [SBVR: Summation of BitVector Representation for Efficient LLM Quantization](https://mlsanity.com/paper/2509.18172/) (2025)
- [Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Models](https://mlsanity.com/paper/2512.21651/) (2025)
- [Diagnostic-Driven Layer-Wise Compensation for Post-Training Quantization of Encoder-Decoder ASR Models](https://mlsanity.com/paper/2601.02455/) (2026)
- [DecompressionLM: Deterministic, Diagnostic, and Zero-Shot Concept Graph Extraction from Language Models](https://mlsanity.com/paper/2602.00377/) (2026)
- [Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuning](https://mlsanity.com/paper/2605.24058/) (2026)
- [Joint Structural Pruning and Mixed-Precision Quantization for LLM Compression](https://mlsanity.com/paper/2606.07819/) (2026)
- [EinSort: Sorting is All We Need for Tensorizing LLM](https://mlsanity.com/paper/2606.08565/) (2026)
- [Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models](https://mlsanity.com/paper/2607.11359/) (2026)
- [Saturation Makes Quantization Error Additive: A Coverage Model with a Certificate](https://mlsanity.com/paper/2607.12266/) (2026)
- [Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Models](https://mlsanity.com/paper/2607.14630/) (2026)
- [One Qubit Can Beat One Bit: Quantum Advantage for Post-Training Quantization](https://mlsanity.com/paper/2608.05240/) (2026)
- [From Sweep to Seam: Interleaved Cross-Block Post-Training Quantization](https://mlsanity.com/paper/2608.09595/) (2026)
- [QuaSAR: Quantization Compensation via Stable Activation-Aware Rank Truncation](https://mlsanity.com/paper/2608.14149/) (2026)

## Abstract (arXiv)

Layer-wise PTQ is a promising technique for compressing large language models (LLMs), due to its simplicity and effectiveness without requiring retraining. However, recent progress in this area is saturating, underscoring the need to revisit its core limitations and explore further improvements. We address this challenge by identifying a key limitation of existing layer-wise PTQ methods: the growth of quantization errors across layers significantly degrades performance, particularly in low-bit regimes. To address this fundamental issue, we propose Quantization Error Propagation (QEP), a general, lightweight, and scalable framework that enhances layer-wise PTQ by explicitly propagating quantization errors and compensating for accumulated errors. QEP also offers a tunable propagation mechanism that prevents overfitting and controls computational overhead, enabling the framework to adapt to various architectures and resource budgets. Extensive experiments on several LLMs demonstrate that QEP-enhanced layer-wise PTQ achieves substantially higher accuracy than existing methods. Notably, the gains are most pronounced in the extremely low-bit quantization regime.