# Zero-shot Adversarial Quantization

arXiv: https://arxiv.org/abs/2103.15263 · HTML: https://mlsanity.com/paper/2103.15263/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 50 references that are in the map.
- First posted on arXiv 29 Mar 2021.

## Builds on (most cited first, 30 of 41)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [PACT: Parameterized Clipping Activation for Quantized Neural Networks](https://mlsanity.com/paper/1805.06085/) (2018)
- [Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion](https://mlsanity.com/paper/1912.08795/) (2019)
- [Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT](https://mlsanity.com/paper/1909.05840/) (2019)
- [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019)
- [DeepPose: Human Pose Estimation via Deep Neural Networks](https://mlsanity.com/paper/1312.4659/) (2013)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [Quantized Convolutional Neural Networks for Mobile Devices](https://mlsanity.com/paper/1512.06473/) (2015)
- [Data-Free Learning of Student Networks](https://mlsanity.com/paper/1904.01186/) (2019)
- [Improving Neural Network Quantization without Retraining using Outlier Channel Splitting](https://mlsanity.com/paper/1901.09504/) (2019)
- [Scalable Methods for 8-bit Training of Neural Networks](https://mlsanity.com/paper/1805.11046/) (2018)
- [Data-Free Knowledge Distillation for Deep Neural Networks](https://mlsanity.com/paper/1710.07535/) (2017)
- [Zero-Shot Knowledge Distillation in Deep Networks](https://mlsanity.com/paper/1905.08114/) (2019)
- [Data-free parameter pruning for Deep Neural Networks](https://mlsanity.com/paper/1507.06149/) (2015)

## Built on by (most cited first, 30 of 44)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation](https://mlsanity.com/paper/2202.07471/) (2022)
- [IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization](https://mlsanity.com/paper/2111.09136/) (2021)
- [Qimera: Data-free Quantization with Synthetic Boundary Supporting Samples](https://mlsanity.com/paper/2111.02625/) (2021)
- [Post-training Quantization for Neural Networks with Provable Guarantees](https://mlsanity.com/paper/2201.11113/) (2022)
- [Data-Free Knowledge Transfer: A Survey](https://mlsanity.com/paper/2112.15278/) (2021)
- [Learning to Retain while Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation](https://mlsanity.com/paper/2302.14290/) (2023)
- [Adaptive Data-Free Quantization](https://mlsanity.com/paper/2303.06869/) (2023)
- [Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning](https://mlsanity.com/paper/2308.07209/) (2023)
- [NAYER: Noisy Layer Data Generation for Efficient and Effective Data-free Knowledge Distillation](https://mlsanity.com/paper/2310.00258/) (2023)
- [It's All In the Teacher: Zero-Shot Quantization Brought Closer to the Teacher](https://mlsanity.com/paper/2203.17008/) (2022)
- [Genie: Show Me the Data for Quantization](https://mlsanity.com/paper/2212.04780/) (2022)
- [Wavelet Feature Maps Compression for Image-to-Image CNNs](https://mlsanity.com/paper/2205.12268/) (2022)
- [CDFKD-MFS: Collaborative Data-free Knowledge Distillation via Multi-level Feature Sharing](https://mlsanity.com/paper/2205.11845/) (2022)
- [Hard Sample Matters a Lot in Zero-Shot Quantization](https://mlsanity.com/paper/2303.13826/) (2023)
- [Rethinking Data-Free Quantization as a Zero-Sum Game](https://mlsanity.com/paper/2302.09572/) (2023)
- [Guided Hybrid Quantization for Object detection in Multimodal Remote Sensing Imagery via One-to-one Self-teaching](https://mlsanity.com/paper/2301.00131/) (2022)
- [Diverse Sample Generation: Pushing the Limit of Generative Data-free Quantization](https://mlsanity.com/paper/2109.00212/) (2021)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [Practical Network Acceleration with Tiny Sets](https://mlsanity.com/paper/2202.07861/) (2022)
- [Fine-grained Data Distribution Alignment for Post-Training Quantization](https://mlsanity.com/paper/2109.04186/) (2021)
- [Data-Free Quantization via Mixed-Precision Compensation without Fine-Tuning](https://mlsanity.com/paper/2307.00498/) (2023)
- [CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification](https://mlsanity.com/paper/2212.02770/) (2022)
- [GenQ: Quantization in Low Data Regimes with Generative Synthetic Data](https://mlsanity.com/paper/2312.05272/) (2023)
- [Zero-shot Quantization: A Comprehensive Survey](https://mlsanity.com/paper/2505.09188/) (2025)
- [Causal-DFQ: Causality Guided Data-free Network Quantization](https://mlsanity.com/paper/2309.13682/) (2023)
- [Relation-Guided Adversarial Learning for Data-free Knowledge Transfer](https://mlsanity.com/paper/2412.11380/) (2024)
- [Quaternion Factorization Machines: A Lightweight Solution to Intricate Feature Interaction Modelling](https://mlsanity.com/paper/2104.01716/) (2021)
- [Long-Range Zero-Shot Generative Deep Network Quantization](https://mlsanity.com/paper/2211.06816/) (2022)
- [Data-free Knowledge Distillation for Fine-grained Visual Categorization](https://mlsanity.com/paper/2404.12037/) (2024)

## Abstract (arXiv)

Model quantization is a promising approach to compress deep neural networks and accelerate inference, making it possible to be deployed on mobile and edge devices. To retain the high performance of full-precision models, most existing quantization methods focus on fine-tuning quantized model by assuming training datasets are accessible. However, this assumption sometimes is not satisfied in real situations due to data privacy and security issues, thereby making these quantization methods not applicable. To achieve zero-short model quantization without accessing training data, a tiny number of quantization methods adopt either post-training quantization or batch normalization statistics-guided data generation for fine-tuning. However, both of them inevitably suffer from low performance, since the former is a little too empirical and lacks training support for ultra-low precision quantization, while the latter could not fully restore the peculiarities of original data and is often low efficient for diverse data generation. To address the above issues, we propose a zero-shot adversarial quantization (ZAQ) framework, facilitating effective discrepancy estimation and knowledge transfer from a full-precision model to its quantized model. This is achieved by a novel two-level discrepancy modeling to drive a generator to synthesize informative and diverse data examples to optimize the quantized model in an adversarial learning fashion. We conduct extensive experiments on three fundamental vision tasks, demonstrating the superiority of ZAQ over the strong zero-shot baselines and validating the effectiveness of its main components. Code is available at <https://git.io/Jqc0y>.