mlsanityData to 14 Oct 2026

COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization

arXiv 2403.07134 ↗First posted 11 Mar 2024cs.LG cs.CVcited by 12

Not on the map. Why?

Abstract (arXiv)

Post-training quantization (PTQ) has emerged as a practical approach to compress large neural networks, making them highly efficient for deployment. However, effectively reducing these models to their low-bit counterparts without compromising the original accuracy remains a key challenge. In this paper, we propose an innovative PTQ algorithm termed COMQ, which sequentially conducts coordinate-wise minimization of the layer-wise reconstruction errors. We consider the widely used integer quantization, where every quantized weight can be decomposed into a shared floating-point scalar and an integer bit-code. Within a fixed layer, COMQ treats all the scaling factor(s) and bit-codes as the variables of the reconstruction error. Every iteration improves this error along a single coordinate while keeping all other variables constant. COMQ is easy to use and requires no hyper-parameter tuning. It instead involves only dot products and rounding operations. We update these variables in a carefully designed greedy order, significantly enhancing the accuracy. COMQ achieves remarkable results in quantizing 4-bit Vision Transformers, with a negligible loss of less than 1% in Top-1 accuracy. In 4-bit INT quantization of convolutional neural networks, COMQ maintains near-lossless accuracy with a minimal drop of merely 0.3% in Top-1 accuracy.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 49 of its 67 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

49 of 67 references.

References · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  8. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  9. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  11. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  12. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
37 more
  1. 2016Pointer Sentinel Mixture Modelscited by 3,096
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  4. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  5. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  6. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 702
  7. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  8. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  9. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  10. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  11. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  12. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  13. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  14. 2021Post-Training Quantization for Vision Transformercited by 278
  15. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  16. 2022Post-training Quantization on Diffusion Modelscited by 267
  17. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 231
  18. 2015Coordinate Descent Algorithmscited by 219
  19. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 180
  20. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 179
  21. 2023PTQD: Accurate Post-Training Quantization for Diffusion Modelscited by 170
  22. 2021FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformercited by 151
  23. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  24. 2020Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programmingcited by 92
  25. 2020Generative Low-bitwidth Data Free Quantizationcited by 88
  26. 2019Blockwisely Supervised Neural Architecture Search with Knowledge Distillationcited by 81
  27. 2023Stable and low-precision training for large-scale vision-language modelscited by 75
  28. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  29. 2023Towards Accurate Post-Training Quantization for Vision Transformercited by 71
  30. 2023FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantizationcited by 48
  31. 2022Post-training Quantization for Neural Networks with Provable Guaranteescited by 33
  32. 2023QuantEase: Optimization-based Quantization for Language Modelscited by 29
  33. 2018BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weightscited by 24
  34. 2018Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networkscited by 21
  35. 2016Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detectioncited by 8
  36. 2016Fixed-point Factorized Networkscited by 5
  37. 2023Feature Affinity Assisted Knowledge Distillation and Quantization of Deep Neural Networks on Label-Free Datacited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 4.

12 papers.

  1. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformerscited by 79
  2. 2024MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantizationcited by 28
  3. 2024Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Surveycited by 13
  4. 2025FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformerscited by 6
  5. 2025CLoQ: Enhancing Fine-Tuning of Quantized LLMs via Calibrated LoRA Initializationcited by 5
  6. 2025Beacon: Post-Training Quantization with Integrated Grid Selectioncited by 3
  7. 2026Optimal Post-Training Quantization Scales and Where to Find Themcited by 3
  8. 2025Eigen Neural Network: Unlocking Generalizable Vision with Eigenbasiscited by 0
  9. 2025Binary Quadratic Quantization: Beyond First-Order Quantization for Real-Valued Matrix Compressioncited by 0
  10. 2026RPIQ: Residual-Projected Multi-Collaboration Closed-Loop and Single Instance Quantization for Visually Impaired Assistancecited by 0
  11. 2026Two-Stage Grid Optimization for Group-wise Quantization of LLMscited by 0
  12. 2026Weight Group-wise Post-Training Quantization for Medical Foundation Modelcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Benchmarking the Reliability of Post-training Quantization: a Particular Focus on Worst-case Performance
  2. 2025Pack-PTQ: Advancing Post-training Quantization of Neural Networks by Pack-wise Reconstruction
  3. 2025Efficiently Training A Flat Neural Network Before It has been Quantizated
  4. 2021Post-Training Sparsity-Aware Quantization
  5. 2025Provable Post-Training Quantization: Theoretical Analysis of OPTQ and Qronos
  6. 2024Progressive Fine-to-Coarse Reconstruction for Accurate Low-Bit Post-Training Quantization in Vision Transformers
  7. 2025Sensitivity-Aware Post-Training Quantization for Deep Neural Networks
  8. 2026TR-PTQ: High-Accuracy Integer-Only Transformer Post Training Quantization via Taylor Region Reformulation