# IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization

arXiv: https://arxiv.org/abs/2111.09136 · HTML: https://mlsanity.com/paper/2111.09136/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 51 references that are in the map.
- First posted on arXiv 17 Nov 2021.

## Builds on (most cited first, 30 of 44)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction](https://mlsanity.com/paper/2102.05426/) (2021)
- [Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion](https://mlsanity.com/paper/1912.08795/) (2019)
- [Tutorial on Variational Autoencoders](https://mlsanity.com/paper/1606.05908/) (2016)
- [Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration](https://mlsanity.com/paper/1811.00250/) (2018)
- [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019)
- [HRank: Filter Pruning using High-Rank Feature Map](https://mlsanity.com/paper/2002.10179/) (2020)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks](https://mlsanity.com/paper/1908.05033/) (2019)
- [Forward and Backward Information Retention for Accurate Binary Neural Networks](https://mlsanity.com/paper/1909.10788/) (2019)
- [Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss](https://mlsanity.com/paper/1808.05779/) (2018)
- [Quantization Networks](https://mlsanity.com/paper/1911.09464/) (2019)
- [Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networks](https://mlsanity.com/paper/1909.13144/) (2019)
- [Training Binary Neural Networks with Real-to-Binary Convolutions](https://mlsanity.com/paper/2003.11535/) (2020)
- [Generative Low-bitwidth Data Free Quantization](https://mlsanity.com/paper/2003.03603/) (2020)
- [Network Quantization with Element-wise Gradient Scaling](https://mlsanity.com/paper/2104.00903/) (2021)
- [Data-Free Network Quantization With Adversarial Knowledge Distillation](https://mlsanity.com/paper/2005.04136/) (2020)
- [Post-Training Piecewise Linear Quantization for Deep Neural Networks](https://mlsanity.com/paper/2002.00104/) (2020)
- [Rotated Binary Neural Network](https://mlsanity.com/paper/2009.13055/) (2020)

## Built on by (most cited first, 30 of 43)

- [Post-training Quantization on Diffusion Models](https://mlsanity.com/paper/2211.15736/) (2022)
- [PD-Quant: Post-Training Quantization based on Prediction Difference Metric](https://mlsanity.com/paper/2212.07048/) (2022)
- [Patch Similarity Aware Data-Free Quantization for Vision Transformers](https://mlsanity.com/paper/2203.02250/) (2022)
- [Towards Accurate Post-training Quantization for Diffusion Models](https://mlsanity.com/paper/2305.18723/) (2023)
- [PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformers](https://mlsanity.com/paper/2209.05687/) (2022)
- [Data-Free Knowledge Transfer: A Survey](https://mlsanity.com/paper/2112.15278/) (2021)
- [Adaptive Data-Free Quantization](https://mlsanity.com/paper/2303.06869/) (2023)
- [Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning](https://mlsanity.com/paper/2308.07209/) (2023)
- [Dynamic Dual Trainable Bounds for Ultra-low Precision Super-Resolution Networks](https://mlsanity.com/paper/2203.03844/) (2022)
- [I&S-ViT: An Inclusive & Stable Method for Pushing the Limit of Post-Training ViTs Quantization](https://mlsanity.com/paper/2311.10126/) (2023)
- [Genie: Show Me the Data for Quantization](https://mlsanity.com/paper/2212.04780/) (2022)
- [CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTs](https://mlsanity.com/paper/2407.05266/) (2024)
- [Hard Sample Matters a Lot in Zero-Shot Quantization](https://mlsanity.com/paper/2303.13826/) (2023)
- [Rethinking Data-Free Quantization as a Zero-Sum Game](https://mlsanity.com/paper/2302.09572/) (2023)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [MimiQ: Low-Bit Data-Free Quantization of Vision Transformers with Encouraging Inter-Head Attention Similarity](https://mlsanity.com/paper/2407.20021/) (2024)
- [GenQ: Quantization in Low Data Regimes with Generative Synthetic Data](https://mlsanity.com/paper/2312.05272/) (2023)
- [Zero-shot Quantization: A Comprehensive Survey](https://mlsanity.com/paper/2505.09188/) (2025)
- [REx: Data-Free Residual Quantization Error Expansion](https://mlsanity.com/paper/2203.14645/) (2022)
- [Causal-DFQ: Causality Guided Data-free Network Quantization](https://mlsanity.com/paper/2309.13682/) (2023)
- [Relation-Guided Adversarial Learning for Data-free Knowledge Transfer](https://mlsanity.com/paper/2412.11380/) (2024)
- [Sharpness-Aware Data Generation for Zero-shot Quantization](https://mlsanity.com/paper/2510.07018/) (2025)
- [Long-Range Zero-Shot Generative Deep Network Quantization](https://mlsanity.com/paper/2211.06816/) (2022)
- [Robustness-Guided Image Synthesis for Data-Free Quantization](https://mlsanity.com/paper/2310.03661/) (2023)
- [Privacy-Preserving SAM Quantization for Efficient Edge Intelligence in Healthcare](https://mlsanity.com/paper/2410.01813/) (2024)
- [Augmenting Hessians with Inter-Layer Dependencies for Mixed-Precision Post-Training Quantization](https://mlsanity.com/paper/2306.04879/) (2023)
- [Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformers](https://mlsanity.com/paper/2412.16553/) (2024)
- [Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training](https://mlsanity.com/paper/2211.08544/) (2022)
- [ACQ: Improving Generative Data-free Quantization Via Attention Correction](https://mlsanity.com/paper/2301.07266/) (2023)
- [Data Generation for Hardware-Friendly Post-Training Quantization](https://mlsanity.com/paper/2410.22110/) (2024)

## Abstract (arXiv)

Learning to synthesize data has emerged as a promising direction in zero-shot quantization (ZSQ), which represents neural networks by low-bit integer without accessing any of the real data. In this paper, we observe an interesting phenomenon of intra-class heterogeneity in real data and show that existing methods fail to retain this property in their synthetic images, which causes a limited performance increase. To address this issue, we propose a novel zero-shot quantization method referred to as IntraQ. First, we propose a local object reinforcement that locates the target objects at different scales and positions of the synthetic images. Second, we introduce a marginal distance constraint to form class-related features distributed in a coarse area. Lastly, we devise a soft inception loss which injects a soft prior label to prevent the synthetic images from being overfitting to a fixed object. Our IntraQ is demonstrated to well retain the intra-class heterogeneity in the synthetic images and also observed to perform state-of-the-art. For example, compared to the advanced ZSQ, our IntraQ obtains 9.17% increase of the top-1 accuracy on ImageNet when all layers of MobileNetV1 are quantized to 4-bit. Code is at https://github.com/zysxmu/IntraQ.