mlsanityData to 7 Oct 2026

Generative Zero-shot Network Quantization

arXiv 2101.08430 ↗First posted 21 Jan 2021cs.CV cs.AIcited by 15

Not on the map. Why?

Abstract (arXiv)

Convolutional neural networks are able to learn realistic image priors from numerous training samples in low-level image generation and restoration. We show that, for high-level image recognition tasks, we can further reconstruct "realistic" images of each category by leveraging intrinsic Batch Normalization (BN) statistics without any training data. Inspired by the popular VAE/GAN methods, we regard the zero-shot optimization process of synthetic images as generative modeling to match the distribution of BN statistics. The generated images serve as a calibration set for the following zero-shot network quantizations. Our method meets the needs for quantizing models based on sensitive information, textite.g., due to privacy concerns, no data is available. Extensive experiments on benchmark datasets show that, with the help of generated data, our approach consistently outperforms existing data-free quantization methods.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 55 of its 81 references that are in the map. First posted on arXiv 21 Jan 2021.

Builds on

55 of 81 references.

References · 55

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  6. 2016Improved Techniques for Training GANscited by 2,938
  7. 2016Deep Learning with Differential Privacycited by 2,682
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  9. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  10. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  11. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  12. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
38 more
  1. 2017Deep Image Priorcited by 1,097
  2. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  3. 2018Self-Attention Generative Adversarial Networkscited by 809
  4. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  5. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  6. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  7. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  8. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  9. 2019Dreaming to Distill: Data-free Knowledge Transfer via DeepInversioncited by 419
  10. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  11. 2016Tutorial on Variational Autoencoderscited by 404
  12. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  13. 2015Deep Learning with Limited Numerical Precisioncited by 368
  14. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  15. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  16. 2018A Note on the Inception Scorecited by 225
  17. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  18. 2016Trained Ternary Quantizationcited by 203
  19. 2019Data-Free Learning of Student Networkscited by 203
  20. 2016Ternary Weight Networkscited by 195
  21. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  22. 2019What Do Compressed Deep Neural Networks Forget?cited by 164
  23. 2020Characterising Bias in Compressed Modelscited by 162
  24. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 150
  25. 2017Towards Accurate Binary Convolutional Neural Networkcited by 148
  26. 2017Data-Free Knowledge Distillation for Deep Neural Networkscited by 136
  27. 2019Zero-shot Knowledge Transfer via Adversarial Belief Matchingcited by 129
  28. 2019Zero-Shot Knowledge Distillation in Deep Networkscited by 119
  29. 2020Training Binary Neural Networks with Real-to-Binary Convolutionscited by 111
  30. 2020MAZE: Data-Free Model Stealing Attack Using Zeroth-Order Gradient Estimationcited by 99
  31. 2020Generative Low-bitwidth Data Free Quantizationcited by 87
  32. 2020Data-Free Network Quantization With Adversarial Knowledge Distillationcited by 79
  33. 2018How good is my GAN?cited by 74
  34. 2016Generative Multi-Adversarial Networkscited by 60
  35. 20158-Bit Approximations for Parallelism in Deep Learningcited by 56
  36. 2019The Knowledge Within: Methods for Data-Free Model Compressioncited by 54
  37. 2019Batch-Shaping for Learning Conditional Channel Gated Networkscited by 42
  38. 2018Post-training 4-bit quantization of convolution networks for rapid-deploymentcited by 38

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 2; 2023: 5; 2024: 0; 2025: 1; 2026: 1.

15 papers.

Sort:

By area:

  1. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  2. 2022Post-training Quantization on Diffusion Modelscited by 262
  3. 2021IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantizationcited by 43
  4. 2021Data-Free Knowledge Transfer: A Surveycited by 31
  5. 2021Distribution-sensitive Information Retention for Accurate Binary Neural Networkcited by 25
  6. 2021Integer-only Zero-shot Quantization for Efficient Speech Recognitioncited by 19
  7. 2023Hard Sample Matters a Lot in Zero-Shot Quantizationcited by 15
  8. 2021Diverse Sample Generation: Pushing the Limit of Generative Data-free Quantizationcited by 11
  9. 2023Data-Free Quantization via Mixed-Precision Compensation without Fine-Tuningcited by 9
  10. 2022Long-Range Zero-Shot Generative Deep Network Quantizationcited by 4
  11. 2023BOMP-NAS: Bayesian Optimization Mixed Precision NAScited by 4
  12. 2025Sparse Model Inversion: Efficient Inversion of Vision Transformers for Data-Free Applicationscited by 4
3 more
  1. 2023ACQ: Improving Generative Data-free Quantization Via Attention Correctioncited by 2
  2. 2023On-Chip Hardware-Aware Quantization for Mixed Precision Neural Networkscited by 1
  3. 2026Toward Multi-Domain and Long-Tailed Quantization via Feature Alignment and Scalingcited by 0