mlsanityData to 14 Oct 2026

BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction

arXiv 2102.05426 ↗First posted 10 Feb 2021cs.LG cs.CVcited by 479

Not on the map. Why?

Cited by 63 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We study the challenging task of neural network quantization without end-to-end retraining, called Post-training Quantization (PTQ). PTQ usually requires a small subset of training data but produces less powerful quantized models than Quantization-Aware Training (QAT). In this work, we propose a novel PTQ framework, dubbed BRECQ, which pushes the limits of bitwidth in PTQ down to INT2 for the first time. BRECQ leverages the basic building blocks in neural networks and reconstructs them one-by-one. In a comprehensive theoretical study of the second-order error, we show that BRECQ achieves a good balance between cross-layer dependency and generalization error. To further employ the power of quantization, the mixed precision technique is incorporated in our framework by approximating the inter-layer and intra-layer sensitivity. Extensive experiments on various handcrafted and searched neural architectures are conducted for both image classification and object detection tasks. And for the first time we prove that, without bells and whistles, PTQ can attain 4-bit ResNet and MobileNetV2 comparable with QAT and enjoy 240 times faster production of quantized models. Codes are available at https://github.com/yhhhli/BRECQ.

Key facts. Cited by 479 papers in the map (data through 14 Oct 2026). Builds on 31 of its 41 references that are in the map. First posted on arXiv 10 Feb 2021.

Builds on

31 of 41 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  6. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  7. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  8. 2020Designing Network Design Spacescited by 960
  9. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  10. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 796
  11. 2019Learned Step Size Quantizationcited by 628
  12. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
19 more
  1. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  2. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  3. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  4. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
  5. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  6. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  7. 2018Model compression via distillation and quantizationcited by 289
  8. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  9. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 238
  10. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 221
  11. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 193
  12. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 148
  13. 2017Practical Gauss-Newton Optimisation for Deep Learningcited by 142
  14. 2017Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networkscited by 109
  15. 2019Loss Aware Post-training Quantizationcited by 97
  16. 2018Faster gaze prediction with dense networks and Fisher pruningcited by 94
  17. 2020Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programmingcited by 92
  18. 2018Generalization Error in Deep Learningcited by 47
  19. 2019Few Shot Network Compression via Cross Distillationcited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 20; 2022: 46; 2023: 75; 2024: 113; 2025: 114; 2026: 111.

479 papers.

Sort:

By area:

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  2. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  3. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  4. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  5. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  6. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  7. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  8. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  9. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  10. 2023Q-Diffusion: Quantizing Diffusion Modelscited by 273
  11. 2022Post-training Quantization on Diffusion Modelscited by 267
  12. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
38 more
  1. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 181
  2. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 180
  3. 2021PTQ4ViT: Post-training quantization for vision transformers with twin uniform quantizationcited by 179
  4. 2023PTQD: Accurate Post-Training Quantization for Diffusion Modelscited by 170
  5. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  6. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 151
  7. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 144
  8. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  9. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  10. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127
  11. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 116
  12. 2024ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generationcited by 115
  13. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 114
  14. 2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformerscited by 94
  15. 2024ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identificationcited by 93
  16. 2022FP8 Quantization: The Power of the Exponentcited by 88
  17. 2022COIN++: Neural Compression Across Modalitiescited by 82
  18. 2023Temporal Dynamic Quantization for Diffusion Modelscited by 81
  19. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformerscited by 79
  20. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Modelscited by 78
  21. 2024KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantizationcited by 76
  22. 2022PD-Quant: Post-Training Quantization based on Prediction Difference Metriccited by 73
  23. 2023LLM-FP4: 4-Bit Floating-Point Quantized Transformerscited by 69
  24. 2023TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Modelscited by 69
  25. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 67
  26. 2022A Comprehensive Survey on Model Quantization for Deep Neural Networks in Image Classificationcited by 65
  27. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 64
  28. 2021Diversifying Sample Generation for Accurate Data-Free Quantizationcited by 63
  29. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scalingcited by 61
  30. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 60
  31. 2022Masked Wavelet Representation for Compact Neural Radiance Fieldscited by 57
  32. 2022Patch Similarity Aware Data-Free Quantization for Vision Transformerscited by 56
  33. 2022SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximationcited by 55
  34. 2023Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMscited by 50
  35. 2023Intriguing Properties of Quantization at Scalecited by 49
  36. 2024QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuningcited by 49
  37. 2023FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantizationcited by 48
  38. 2023Towards Accurate Post-training Quantization for Diffusion Modelscited by 47

50 most cited of 479 All 479 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021A White Paper on Neural Network Quantization
  2. 2021BMPQ: Bit-Gradient Sensitivity Driven Mixed-Precision Quantization of DNNs from Scratch
  3. 2023Bag of Tricks with Quantized Convolutional Neural Networks for image classification
  4. 2023RepQ: Generalizing Quantization-Aware Training for Re-Parametrized Architectures
  5. 2026Understanding Quantization-Aware Training: Gradients at Quantized Weights Bias to the Low-Loss Basin
  6. 2025Provable Post-Training Quantization: Theoretical Analysis of OPTQ and Qronos
  7. 2021HPTQ: Hardware-Friendly Post Training Quantization
  8. 2023A Practical Mixed Precision Algorithm for Post-Training Quantization