mlsanityData to 7 Oct 2026

QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization

arXiv 2203.05740 ↗First posted 11 Mar 2022cs.CV cs.AIcited by 175

Not on the map. Why?

Cited by 27 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recently, post-training quantization (PTQ) has driven much attention to produce efficient neural networks without long-time retraining. Despite its low cost, current PTQ works tend to fail under the extremely low-bit setting. In this study, we pioneeringly confirm that properly incorporating activation quantization into the PTQ reconstruction benefits the final accuracy. To deeply understand the inherent reason, a theoretical framework is established, indicating that the flatness of the optimized low-bit model on calibration and test data is crucial. Based on the conclusion, a simple yet effective approach dubbed as QDROP is proposed, which randomly drops the quantization of activations during PTQ. Extensive experiments on various tasks including computer vision (image classification, object detection) and natural language processing (text classification and question answering) prove its superiority. With QDROP, the limit of PTQ is pushed to the 2-bit activation for the first time and the accuracy boost can be up to 51.49%. Without bells and whistles, QDROP establishes a new state of the art for PTQ. Our code is available at https://github.com/wimh966/QDrop and has been integrated into MQBench (https://github.com/ModelTC/MQBench)

Key facts. Cited by 175 papers in the map (data through 7 Oct 2026). Builds on 37 of its 50 references that are in the map. First posted on arXiv 11 Mar 2022.

Builds on

37 of 50 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  9. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  10. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  11. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  12. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
25 more
  1. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  2. 2020Designing Network Design Spacescited by 954
  3. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  4. 2019Learned Step Size Quantizationcited by 604
  5. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  6. 2017Exploring Generalization in Deep Learningcited by 595
  7. 2019Fantastic Generalization Measures and Where to Find Themcited by 507
  8. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  9. 2017Sharp Minima Can Generalize For Deep Netscited by 462
  10. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  11. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  12. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
  13. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  14. 2020BinaryBERT: Pushing the Limit of BERT Quantizationcited by 175
  15. 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
  16. 2020Training with Quantization Noise for Extreme Model Compressioncited by 145
  17. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 143
  18. 2019Loss Aware Post-training Quantizationcited by 94
  19. 2020Regularizing Neural Networks via Adversarial Model Perturbationcited by 78
  20. 2021Diversifying Sample Generation for Accurate Data-Free Quantizationcited by 63
  21. 2019SWALP : Stochastic Weight Averaging in Low-Precision Trainingcited by 40
  22. 2021MQBench: Towards Reproducible and Deployable Model Quantization Benchmarkcited by 38
  23. 2020MixMix: All You Need for Data-Free Compression Are Feature and Data Mixingcited by 29
  24. 2020Once Quantization-Aware Training: High Performance Extremely Low-bit Architecture Searchcited by 26
  25. 2021Is In-Domain Data Really Needed? A Pilot Study on Cross-Domain Calibration for Network Quantizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 30; 2024: 56; 2025: 49; 2026: 30.

175 papers.

Sort:

By area:

  1. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  2. 2023PTQD: Accurate Post-Training Quantization for Diffusion Modelscited by 167
  3. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 134
  4. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 113
  5. 2023Temporal Dynamic Quantization for Diffusion Modelscited by 81
  6. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Modelscited by 77
  7. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformerscited by 77
  8. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  9. 2023TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Modelscited by 67
  10. 2023LLM-FP4: 4-Bit Floating-Point Quantized Transformerscited by 65
  11. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 64
  12. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 63
38 more
  1. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 62
  2. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 54
  3. 2023Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMscited by 49
  4. 2024QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuningcited by 48
  5. 2023Towards Accurate Post-training Quantization for Diffusion Modelscited by 46
  6. 2023FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantizationcited by 46
  7. 2024PTQ4SAM: Post-Training Quantization for Segment Anythingcited by 32
  8. 2024LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkitcited by 32
  9. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
  10. 2024LiDAR-PTQ: Post-Training Quantization for Point Cloud 3D Object Detectioncited by 29
  11. 2024AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizercited by 28
  12. 2024MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantizationcited by 26
  13. 2024Quantization without Tearscited by 24
  14. 2024Instance-Aware Group Quantization for Vision Transformerscited by 23
  15. 2024Towards Accurate Post-Training Quantization of Vision Transformers via Error Reductioncited by 23
  16. 2024MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Modelscited by 20
  17. 2023I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantizationcited by 19
  18. 2024I-LLM: Efficient Integer-Only Inference for Fully-Quantized Low-Bit Large Language Modelscited by 19
  19. 2024DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformerscited by 19
  20. 2022Genie: Show Me the Data for Quantizationcited by 17
  21. 2023Solving Oscillation Problem in Post-Training Quantization Through a Theoretical Perspectivecited by 17
  22. 2025APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformerscited by 17
  23. 2025FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximationcited by 16
  24. 2025Quantized Visual Geometry Grounded Transformercited by 16
  25. 2024Advancing Multimodal Large Language Models with Quantization-Aware Scale Learning for Efficient Adaptationcited by 15
  26. 2024Nearest is Not Dearest: Towards Practical Defense against Quantization-conditioned Backdoor Attackscited by 14
  27. 2024Information Entropy Guided Height-aware Histogram for Quantization-friendly Pillar Feature Encodercited by 14
  28. 2024QVD: Post-training Quantization for Video Diffusion Modelscited by 14
  29. 2025CacheQuant: Comprehensively Accelerated Diffusion Modelscited by 13
  30. 2024COMQ: A Backpropagation-Free Algorithm for Post-Training Quantizationcited by 12
  31. 2025MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillationcited by 12
  32. 2025S2Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillationcited by 12
  33. 2023Squeezing Large-Scale Diffusion Models for Mobilecited by 11
  34. 2024BinaryDM: Accurate Weight Binarization for Efficient Diffusion Modelscited by 11
  35. 2024LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matricescited by 11
  36. 2025DGQ: Distribution-Aware Group Quantization for Text-to-Image Diffusion Modelscited by 11
  37. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  38. 2023EQ-Net: Elastic Quantization Neural Networkscited by 10

50 most cited of 175 All 175 (JSON)