# Robust Quantization: One Model to Rule Them All

arXiv: https://arxiv.org/abs/2002.07686 · HTML: https://mlsanity.com/paper/2002.07686/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2020.

## Built on by (most cited first, 30 of 63)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models](https://mlsanity.com/paper/2209.13325/) (2022)
- [Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing](https://mlsanity.com/paper/2306.12929/) (2023)
- [FlatQuant: Flatness Matters for LLM Quantization](https://mlsanity.com/paper/2410.09426/) (2024)
- [ALISA: Accelerating Large Language Model Inference via Sparsity-Aware KV Caching](https://mlsanity.com/paper/2403.17312/) (2024)
- [Outlier-Efficient Hopfield Layers for Large Transformer-Based Models](https://mlsanity.com/paper/2404.03828/) (2024)
- [NIPQ: Noise proxy-based Integrated Pseudo-Quantization](https://mlsanity.com/paper/2206.00820/) (2022)
- [SDQ: Stochastic Differentiable Quantization with Mixed Precision](https://mlsanity.com/paper/2206.04459/) (2022)
- [Pareto-Optimal Quantized ResNet Is Mostly 4-bit](https://mlsanity.com/paper/2105.03536/) (2021)
- [Anchor-based Plain Net for Mobile Image Super-Resolution](https://mlsanity.com/paper/2105.09750/) (2021)
- [Mitigating the Impact of Outlier Channels for Language Model Quantization with Activation Regularization](https://mlsanity.com/paper/2404.03605/) (2024)
- [Towards Accurate Post-Training Quantization of Vision Transformers via Error Reduction](https://mlsanity.com/paper/2407.06794/) (2024)
- [BatchQuant: Quantized-for-all Architecture Search with Robust Quantizer](https://mlsanity.com/paper/2105.08952/) (2021)
- [Post-Training Sparsity-Aware Quantization](https://mlsanity.com/paper/2105.11010/) (2021)
- [Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precision](https://mlsanity.com/paper/2307.00331/) (2023)
- [Do All MobileNets Quantize Poorly? Gaining Insights into the Effect of Quantization on Depthwise Separable Convolutional Networks Through the Eyes of Multi-scale Distributional Dynamics](https://mlsanity.com/paper/2104.11849/) (2021)
- [Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices](https://mlsanity.com/paper/2206.10844/) (2022)
- [KurTail : Kurtosis-based LLM Quantization](https://mlsanity.com/paper/2503.01483/) (2025)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [EQ-Net: Elastic Quantization Neural Networks](https://mlsanity.com/paper/2308.07650/) (2023)
- [Fast and Low-Cost Genomic Foundation Models via Outlier Removal](https://mlsanity.com/paper/2505.00598/) (2025)
- [Value-State Gated Attention for Mitigating Extreme-Token Phenomena in Transformers](https://mlsanity.com/paper/2510.09017/) (2025)
- [QGen: On the Ability to Generalize in Quantization Aware Training](https://mlsanity.com/paper/2404.11769/) (2024)
- [Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications](https://mlsanity.com/paper/2508.15008/) (2025)
- [MBQuant: A Novel Multi-Branch Topology Method for Arbitrary Bit-width Network Quantization](https://mlsanity.com/paper/2305.08117/) (2023)
- [Post-Training Quantization for Re-parameterization via Coarse & Fine Weight Splitting](https://mlsanity.com/paper/2312.10588/) (2023)
- [LoopQ: Quantization for Looped Language Models](https://mlsanity.com/paper/2605.16343/) (2026)
- [2-in-1 Accelerator: Enabling Random Precision Switch for Winning Both Adversarial Robustness and Efficiency](https://mlsanity.com/paper/2109.05223/) (2021)
- [FedComLoc: Communication-Efficient Distributed Training of Sparse and Quantized Models](https://mlsanity.com/paper/2403.09904/) (2024)
- [Bimodal Distributed Binarized Neural Networks](https://mlsanity.com/paper/2204.02004/) (2022)

## Abstract (arXiv)

Neural network quantization methods often involve simulating the quantization process during training, making the trained model highly dependent on the target bit-width and precise way quantization is performed. Robust quantization offers an alternative approach with improved tolerance to different classes of data-types and quantization policies. It opens up new exciting applications where the quantization process is not static and can vary to meet different circumstances and implementations. To address this issue, we propose a method that provides intrinsic robustness to the model against a broad range of quantization processes. Our method is motivated by theoretical arguments and enables us to store a single generic model capable of operating at various bit-widths and quantization policies. We validate our method's effectiveness on different ImageNet models.