# Quantization without Tears

arXiv: https://arxiv.org/abs/2411.13918 · HTML: https://mlsanity.com/paper/2411.13918/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 62 references that are in the map.
- First posted on arXiv 21 Nov 2024.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction](https://mlsanity.com/paper/2102.05426/) (2021)
- [SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression](https://mlsanity.com/paper/2306.03078/) (2023)

## Built on by (most cited first, 25 of 25)

- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers](https://mlsanity.com/paper/2506.11784/) (2025)
- [TWEO: Transformers Without Extreme Outliers Enables FP8 Training And Quantization For Dummies](https://mlsanity.com/paper/2511.23225/) (2025)
- [QwT-v2: Practical, Effective and Efficient Post-Training Quantization](https://mlsanity.com/paper/2505.20932/) (2025)
- [TernaryCLIP: Efficiently Compressing Vision-Language Models with Ternary Weights and Distilled Knowledge](https://mlsanity.com/paper/2510.21879/) (2025)
- [ZipMoE: Efficient On-Device MoE Serving via Lossless Compression and Cache-Affinity Scheduling](https://mlsanity.com/paper/2601.21198/) (2026)
- [A Comprehensive Evaluation on Quantization Techniques for Large Language Models](https://mlsanity.com/paper/2507.17417/) (2025)
- [Less Precise Can Be More Reliable: A Systematic Evaluation of Quantization's Impact on VLMs Beyond Accuracy](https://mlsanity.com/paper/2509.21173/) (2025)
- [MVA 2025 Small Multi-Object Tracking for Spotting Birds Challenge: Dataset, Methods, and Results](https://mlsanity.com/paper/2507.12832/) (2025)
- [PTQAT: A Hybrid Parameter-Efficient Quantization Algorithm for 3D Perception Tasks](https://mlsanity.com/paper/2508.10557/) (2025)
- [Tail-Aware Post-Training Quantization for 3D Geometry Models](https://mlsanity.com/paper/2602.01741/) (2026)
- [Post-Training Quantization via Residual Truncation and Zero Suppression for Diffusion Models](https://mlsanity.com/paper/2509.26436/) (2025)
- [LSGQuant: Layer-Sensitivity Guided Quantization for One-Step Diffusion Real-World Video Super-Resolution](https://mlsanity.com/paper/2602.03182/) (2026)
- [Nonlinear Bipolar Compensation: Handling Outliers in Post-Training Quantization](https://mlsanity.com/paper/2605.16423/) (2026)
- [SAB-LVLM: Significance-Aware Binarization for Large Vision-Language Models](https://mlsanity.com/paper/2607.01876/) (2026)
- [YOLOv8-SMOT: An Efficient and Robust Framework for Real-Time Small Object Tracking via Slice-Assisted Training and Adaptive Association](https://mlsanity.com/paper/2507.12087/) (2025)
- [Integrating Pruning with Quantization for Efficient Deep Neural Networks Compression](https://mlsanity.com/paper/2509.04244/) (2025)
- [Assessing the Potential for Catastrophic Failure in Dynamic Post-Training Quantization](https://mlsanity.com/paper/2510.02457/) (2025)
- [Toward Multi-Domain and Long-Tailed Quantization via Feature Alignment and Scaling](https://mlsanity.com/paper/2606.04920/) (2026)
- [Efficient Tuning Before Low-Bit Post-Training Quantization for Stochastic Gradient Descent-optimized Models](https://mlsanity.com/paper/2607.11359/) (2026)
- [QuaSAR: Quantization Compensation via Stable Activation-Aware Rank Truncation](https://mlsanity.com/paper/2608.14149/) (2026)
- [SandwichQuant: Which Parameters Matter Before and After Quantization?](https://mlsanity.com/paper/2608.24173/) (2026)
- [Mind the Approximation: Fisher-Weighted SVD Compression for ViTs](https://mlsanity.com/paper/2609.07155/) (2026)
- [Precision As You Need: Stochastic Computing Is a Dense Adaptive Quantizer](https://mlsanity.com/paper/2609.32922/) (2026)
- [GLF-Q: Global-Local Feature-based Quantization for Vision Transformers](https://mlsanity.com/paper/2609.34564/) (2026)

## Abstract (arXiv)

Deep neural networks, while achieving remarkable success across diverse tasks, demand significant resources, including computation, GPU memory, bandwidth, storage, and energy. Network quantization, as a standard compression and acceleration technique, reduces storage costs and enables potential inference acceleration by discretizing network weights and activations into a finite set of integer values. However, current quantization methods are often complex and sensitive, requiring extensive task-specific hyperparameters, where even a single misconfiguration can impair model performance, limiting generality across different models and tasks. In this paper, we propose Quantization without Tears (QwT), a method that simultaneously achieves quantization speed, accuracy, simplicity, and generality. The key insight of QwT is to incorporate a lightweight additional structure into the quantized network to mitigate information loss during quantization. This structure consists solely of a small set of linear layers, keeping the method simple and efficient. More importantly, it provides a closed-form solution, allowing us to improve accuracy effortlessly under 2 minutes. Extensive experiments across various vision, language, and multimodal tasks demonstrate that QwT is both highly effective and versatile. In fact, our approach offers a robust solution for network quantization that combines simplicity, accuracy, and adaptability, which provides new insights for the design of novel quantization paradigms. The code is publicly available at https://github.com/wujx2001/QwT