# It's All In the Teacher: Zero-Shot Quantization Brought Closer to the Teacher

arXiv: https://arxiv.org/abs/2203.17008 · HTML: https://mlsanity.com/paper/2203.17008/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 60 of its 73 references that are in the map.
- First posted on arXiv 31 Mar 2022.

## Builds on (most cited first, 30 of 60)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Conditional Image Synthesis With Auxiliary Classifier GANs](https://mlsanity.com/paper/1610.09585/) (2016)
- [Large Batch Training of Convolutional Networks](https://mlsanity.com/paper/1708.03888/) (2017)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion](https://mlsanity.com/paper/1912.08795/) (2019)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [On the Efficacy of Knowledge Distillation](https://mlsanity.com/paper/1910.01348/) (2019)
- [Entropy-SGD: Biasing Gradient Descent Into Wide Valleys](https://mlsanity.com/paper/1611.01838/) (2016)
- [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019)
- [PyHessian: Neural Networks Through the Lens of the Hessian](https://mlsanity.com/paper/1912.07145/) (2019)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [Three Factors Influencing Minima in SGD](https://mlsanity.com/paper/1711.04623/) (2017)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)

## Built on by (most cited first, 20 of 20)

- [PD-Quant: Post-Training Quantization based on Prediction Difference Metric](https://mlsanity.com/paper/2212.07048/) (2022)
- [Adaptive Data-Free Quantization](https://mlsanity.com/paper/2303.06869/) (2023)
- [Genie: Show Me the Data for Quantization](https://mlsanity.com/paper/2212.04780/) (2022)
- [Hard Sample Matters a Lot in Zero-Shot Quantization](https://mlsanity.com/paper/2303.13826/) (2023)
- [Rethinking Data-Free Quantization as a Zero-Sum Game](https://mlsanity.com/paper/2302.09572/) (2023)
- [MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity](https://mlsanity.com/paper/2407.20021/) (2024)
- [GenQ: Quantization in Low Data Regimes with Generative Synthetic Data](https://mlsanity.com/paper/2312.05272/) (2023)
- [Zero-shot Quantization: A Comprehensive Survey](https://mlsanity.com/paper/2505.09188/) (2025)
- [Long-Range Zero-Shot Generative Deep Network Quantization](https://mlsanity.com/paper/2211.06816/) (2022)
- [Robustness-Guided Image Synthesis for Data-Free Quantization](https://mlsanity.com/paper/2310.03661/) (2023)
- [Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformers](https://mlsanity.com/paper/2412.16553/) (2024)
- [ACQ: Improving Generative Data-free Quantization Via Attention Correction](https://mlsanity.com/paper/2301.07266/) (2023)
- [Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models](https://mlsanity.com/paper/2310.13315/) (2023)
- [Data Generation for Hardware-Friendly Post-Training Quantization](https://mlsanity.com/paper/2410.22110/) (2024)
- [Advanced Knowledge Transfer: Refined Feature Distillation for Zero-Shot Quantization in Edge Computing](https://mlsanity.com/paper/2412.19125/) (2024)
- [SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuning](https://mlsanity.com/paper/2603.18423/) (2026)
- [PIPE : Parallelized Inference Through Post-Training Quantization Ensembling of Residual Expansions](https://mlsanity.com/paper/2311.15806/) (2023)
- [DataFreeShield: Defending Adversarial Attacks without Training Data](https://mlsanity.com/paper/2406.15635/) (2024)
- [GranQ: Efficient Channel-wise Quantization via Vectorized Pre-Scaling for Zero-Shot QAT](https://mlsanity.com/paper/2503.18339/) (2025)
- [Toward Multi-Domain and Long-Tailed Quantization via Feature Alignment and Scaling](https://mlsanity.com/paper/2606.04920/) (2026)

## Abstract (arXiv)

Model quantization is considered as a promising method to greatly reduce the resource requirements of deep neural networks. To deal with the performance drop induced by quantization errors, a popular method is to use training data to fine-tune quantized networks. In real-world environments, however, such a method is frequently infeasible because training data is unavailable due to security, privacy, or confidentiality concerns. Zero-shot quantization addresses such problems, usually by taking information from the weights of a full-precision teacher network to compensate the performance drop of the quantized networks. In this paper, we first analyze the loss surface of state-of-the-art zero-shot quantization techniques and provide several findings. In contrast to usual knowledge distillation problems, zero-shot quantization often suffers from 1) the difficulty of optimizing multiple loss terms together, and 2) the poor generalization capability due to the use of synthetic samples. Furthermore, we observe that many weights fail to cross the rounding threshold during training the quantized networks even when it is necessary to do so for better performance. Based on the observations, we propose AIT, a simple yet powerful technique for zero-shot quantization, which addresses the aforementioned two problems in the following way: AIT i) uses a KL distance loss only without a cross-entropy loss, and ii) manipulates gradients to guarantee that a certain portion of weights are properly updated after crossing the rounding thresholds. Experiments show that AIT outperforms the performance of many existing methods by a great margin, taking over the overall state-of-the-art position in the field.