mlsanityData to 7 Oct 2026

Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performance

arXiv 2303.13003 ↗First posted 23 Mar 2023cs.LG cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Post-training quantization (PTQ) is a popular method for compressing deep neural networks (DNNs) without modifying their original architecture or training procedures. Despite its effectiveness and convenience, the reliability of PTQ methods in the presence of some extrem cases such as distribution shift and data noise remains largely unexplored. This paper first investigates this problem on various commonly-used PTQ methods. We aim to answer several research questions related to the influence of calibration set distribution variations, calibration paradigm selection, and data augmentation or sampling strategies on PTQ reliability. A systematic evaluation process is conducted across a wide range of tasks and commonly-used PTQ paradigms. The results show that most existing PTQ methods are not reliable enough in term of the worst-case group performance, highlighting the need for more robust methods. Our findings provide insights for developing PTQ methods that can effectively handle distribution shift scenarios and enable the deployment of quantized DNNs in real-world applications.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 22 of its 37 references that are in the map. First posted on arXiv 23 Mar 2023.

Builds on

22 of 37 references.

References · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  5. 2018Co-teaching: Robust Training of Deep Neural Networks with Extremely Noisy Labelscited by 1,053
  6. 2019Learned Step Size Quantizationcited by 604
  7. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  8. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  9. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  10. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  11. 2021A White Paper on Neural Network Quantizationcited by 453
  12. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
10 more
  1. 2018Learning Models with Uniform Performance via Distributionally Robust Optimizationcited by 334
  2. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 234
  3. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  4. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 175
  5. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 173
  6. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 143
  7. 2020A Survey of Label-noise Representation Learning: Past, Present and Futurecited by 121
  8. 2016Convolutional Neural Networks using Logarithmic Data Representationcited by 97
  9. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  10. 2020EasyQuant: Post-training Quantization via Scale Optimizationcited by 57

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 2; 2026: 1.

8 papers.

  1. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  2. 2023RobustMQ: Benchmarking Robustness of Quantized Modelscited by 19
  3. 2024Investigating the Impact of Quantization on Adversarial Robustnesscited by 9
  4. 2025NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cachecited by 7
  5. 2024Scaling Laws for Post Training Quantized Large Language Modelscited by 4
  6. 2023Towards Clip-Free Quantized Super-Resolution Networks: How to Tame Representative Imagescited by 0
  7. 2025Assessing the Potential for Catastrophic Failure in Dynamic Post-Training Quantizationcited by 0
  8. 2026T3C: Test-Time Tensor Compression with Consistency Guaranteescited by 0