mlsanityData to 14 Oct 2026

Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization

arXiv 2504.09629 ↗First posted 13 Apr 2025cs.LG stat.AP stat.MEcited by 41

Not on the map. Why?

Abstract (arXiv)

Layer-wise PTQ is a promising technique for compressing large language models (LLMs), due to its simplicity and effectiveness without requiring retraining. However, recent progress in this area is saturating, underscoring the need to revisit its core limitations and explore further improvements. We address this challenge by identifying a key limitation of existing layer-wise PTQ methods: the growth of quantization errors across layers significantly degrades performance, particularly in low-bit regimes. To address this fundamental issue, we propose Quantization Error Propagation (QEP), a general, lightweight, and scalable framework that enhances layer-wise PTQ by explicitly propagating quantization errors and compensating for accumulated errors. QEP also offers a tunable propagation mechanism that prevents overfitting and controls computational overhead, enabling the framework to adapt to various architectures and resource budgets. Extensive experiments on several LLMs demonstrate that QEP-enhanced layer-wise PTQ achieves substantially higher accuracy than existing methods. Notably, the gains are most pronounced in the extremely low-bit quantization regime.

Key facts. Cited by 41 papers in the map (data through 14 Oct 2026). Builds on 63 of its 76 references that are in the map. First posted on arXiv 13 Apr 2025.

Builds on

63 of 76 references.

References · 63

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  7. 2016Pointer Sentinel Mixture Modelscited by 3,096
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  9. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  10. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  11. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  12. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
38 more
  1. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  2. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  3. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  4. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  5. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  6. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  7. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  8. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  9. 2021A White Paper on Neural Network Quantizationcited by 472
  10. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  11. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  12. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  13. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 417
  14. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  15. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  16. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  17. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  18. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  19. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  20. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  21. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  22. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  23. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 231
  24. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 202
  25. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 189
  26. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  27. 2019Defensive Quantization: When Efficiency Meets Robustnesscited by 95
  28. 2023PB-LLM: Partially Binarized Large Language Modelscited by 83
  29. 2024OneBit: Towards Extremely Low-bit Large Language Modelscited by 79
  30. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 67
  31. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 64
  32. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 61
  33. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 60
  34. 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelscited by 57
  35. 2024Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Applicationcited by 56
  36. 2018A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Datasetcited by 55
  37. 2023Identifying and Mitigating Vulnerabilities in LLM-Integrated Applicationscited by 54
  38. 2024Low-Rank Adaptation for Foundation Models: A Comprehensive Reviewcited by 37

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 35.

41 papers.

Sort:

By area:

  1. 2025LPCD: Unified Framework from Layer-Wise to Submodule Quantizationcited by 6
  2. 2026CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantizationcited by 6
  3. 2025LoaQ: Layer-wise Output Approximation Quantizationcited by 5
  4. 2025High-Dimensional Learning Dynamics of Quantized Models with Straight-Through Estimatorcited by 4
  5. 2025More Than Bits: Multi-Envelope Double Binary Factorization for Extreme Quantizationcited by 3
  6. 2026OneComp: One-Line Revolution for Generative AI Model Compressioncited by 3
  7. 2026KronQ: LLM Quantization via Kronecker-Factored Hessiancited by 3
  8. 2026D2Quant: Accurate Low-bit Post-Training Weight Quantization for LLMscited by 2
  9. 2026Quantizing Time-Series Models As Dynamical Systems: Trajectory-Based Quantization Sensitivity Scorecited by 2
  10. 2026Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMscited by 2
  11. 2026NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Modelscited by 1
  12. 2026OJBKQ: Objective-Joint Babai-Klein Quantizationcited by 1
29 more
  1. 2026TTQ: Activation-Aware Test-Time Quantization to Accelerate LLM Inference On The Flycited by 1
  2. 2026Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimatecited by 1
  3. 2026OffQ: Taming Structured Outliers in LLM Quantization by Offsettingcited by 1
  4. 2026Why Does Post-Training Quantization Work?cited by 1
  5. 2026Beyond Scalar Sensitivity: Activation-Aware Mixed-Precision LLM Quantization with Cross-Layer Refinementcited by 1
  6. 2025SBVR: Summation of BitVector Representation for Efficient LLM Quantizationcited by 0
  7. 2025Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Modelscited by 0
  8. 2026Diagnostic-Driven Layer-Wise Compensation for Post-Training Quantization of Encoder-Decoder ASR Modelscited by 0
  9. 2026DecompressionLM: Deterministic, Diagnostic, and Zero-Shot Concept Graph Extraction from Language Modelscited by 0
  10. 2026Signs Beat Floats: Low-Rank Double-Binary Adaptation for On-Device Fine-Tuningcited by 0
  11. 2026Joint Structural Pruning and Mixed-Precision Quantization for LLM Compressioncited by 0
  12. 2026EinSort: Sorting is All We Need for Tensorizing LLMcited by 0
  13. 2026Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Modelscited by 0
  14. 2026Saturation Makes Quantization Error Additive: A Coverage Model with a Certificatecited by 0
  15. 2026Cross-Layer Error Compensation and Finite-Sample Feature-Statistics Matching for Extreme Low-Bit Quantization of Large Language Modelscited by 0
  16. 2026One Qubit Can Beat One Bit: Quantum Advantage for Post-Training Quantizationcited by 0
  17. 2026From Sweep to Seam: Interleaved Cross-Block Post-Training Quantizationcited by 0
  18. 2026QuaSAR: Quantization Compensation via Stable Activation-Aware Rank Truncationcited by 0
  19. 2026SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantizationcited by 0
  20. 2026SandwichQuant: Which Parameters Matter Before and After Quantization?cited by 0
  21. 2026Understanding LLM Quantization through Activation-Guided Compensation and Orthogonal Residualscited by 0
  22. 2026Component Type, Not Reconstruction Error, Predicts Attention Quantization Sensitivitycited by 0
  23. 2026Teacher-Anchored Selection of Post-Training Quantized Models under Domain Shiftcited by 0
  24. 2026Depth Laws for the Precision Floor of Trained Neural Networks: Amplification, Residual Scaling, and a Quantization-Aware Training Paradoxcited by 0
  25. 2026JARQ: Joint Alternating Refinement for Quantizationcited by 0
  26. 2026Layerwise Error Attribution for Fast and Robust Mixed-Precision Post-Training Quantizationcited by 0
  27. 2026SAGE: Pushing 2-Bit Vector-Quantized LLMs toward Real-World Usabilitycited by 0
  28. 2026When Does Compensating Inherited Quantization Error Help? Mean-Square Guarantees for Layer-Wise Quantization and Their Limitscited by 0
  29. 2026Revisiting Error Feedback for Model Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Layer-Wise High-Impact Parameter Ratio Optimization in Post-Training Quantization for Large Language Models
  2. 2024Layer-Wise Quantization: A Pragmatic and Effective Method for Quantizing LLMs Beyond Integer Bit-Levels
  3. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models
  4. 2025A Comprehensive Evaluation on Quantization Techniques for Large Language Models
  5. 2026SliderQuant: Accurate Post-Training Quantization for LLMs
  6. 2024Understanding the Difficulty of Low-Precision Post-Training Quantization for LLMs
  7. 2026REAL-Q: E2E LLM Quantization via Dynamic Gradient Descent
  8. 2025PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Models