mlsanityData to 14 Oct 2026

Hard Sample Matters a Lot in Zero-Shot Quantization

arXiv 2303.13826 ↗First posted 24 Mar 2023cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

Zero-shot quantization (ZSQ) is promising for compressing and accelerating deep neural networks when the data for training full-precision models are inaccessible. In ZSQ, network quantization is performed using synthetic samples, thus, the performance of quantized models depends heavily on the quality of synthetic samples. Nonetheless, we find that the synthetic samples constructed in existing ZSQ methods can be easily fitted by models. Accordingly, quantized models obtained by these methods suffer from significant performance degradation on hard samples. To address this issue, we propose HArd sample Synthesizing and Training (HAST). Specifically, HAST pays more attention to hard samples when synthesizing samples and makes synthetic samples hard to fit when training quantized models. HAST aligns features extracted by full-precision and quantized models to ensure the similarity between features extracted by these two models. Extensive experiments show that HAST significantly outperforms existing ZSQ methods, achieving performance comparable to models that are quantized with real data.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 50 of its 55 references that are in the map. First posted on arXiv 24 Mar 2023.

Builds on

50 of 55 references.

References · 50

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2014Going Deeper with Convolutionscited by 4,241
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  9. 2015Fast R-CNNcited by 2,933
  10. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  11. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  12. 2014How transferable are features in deep neural networks?cited by 1,225
38 more
  1. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,077
  2. 2019Learned Step Size Quantizationcited by 628
  3. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  4. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  5. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  6. 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 292
  7. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  8. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
  9. 2019Libra R-CNN: Towards Balanced Learning for Object Detectioncited by 255
  10. 2019Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networkscited by 238
  11. 2019Forward and Backward Information Retention for Accurate Binary Neural Networkscited by 169
  12. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 160
  13. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 148
  14. 2022Federated Class-Incremental Learningcited by 127
  15. 2018Gradient Harmonized Single-stage Detectorcited by 114
  16. 2020Training Binary Neural Networks with Real-to-Binary Convolutionscited by 112
  17. 2022Is Out-of-Distribution Detection Learnable?cited by 109
  18. 2020Generative Low-bitwidth Data Free Quantizationcited by 88
  19. 2021Network Quantization with Element-wise Gradient Scalingcited by 87
  20. 2022Weakly-Supervised Multi-Granularity Map Learning for Vision-and-Language Navigationcited by 86
  21. 2020Rotated Binary Neural Networkcited by 77
  22. 2021Diversifying Sample Generation for Accurate Data-Free Quantizationcited by 63
  23. 2022SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximationcited by 55
  24. 2019The Knowledge Within: Methods for Data-Free Model Compressioncited by 54
  25. 2022Virtual Homogeneity Learning: Defending against Data Heterogeneity in Federated Learningcited by 52
  26. 2020What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentationcited by 50
  27. 2021Zero-shot Adversarial Quantizationcited by 44
  28. 2021IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantizationcited by 43
  29. 2021Qimera: Data-free Quantization with Synthetic Boundary Supporting Samplescited by 40
  30. 2018Structured Binary Neural Networks for Accurate Image Classification and Semantic Segmentationcited by 37
  31. 2021CausalAdv: Adversarial Robustness through the Lens of Causalitycited by 37
  32. 2019Training Quantized Neural Networks with a Full-precision Auxiliary Modulecited by 35
  33. 2021Data-Free Knowledge Transfer: A Surveycited by 31
  34. 2020MixMix: All You Need for Data-Free Compression Are Feature and Data Mixingcited by 29
  35. 2022Learning Active Camera for Multi-Object Navigationcited by 21
  36. 2022It's All In the Teacher: Zero-Shot Quantization Brought Closer to the Teachercited by 20
  37. 2021AutoReCon: Neural Architecture Search-based Reconstruction for Data-free Compressioncited by 17
  38. 2021Generative Zero-shot Network Quantizationcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 7; 2025: 4; 2026: 2.

15 papers.

Sort:

By area:

  1. 2024Enhancing One-Shot Federated Learning Through Data and Ensemble Co-Boostingcited by 27
  2. 2024EDA-DM: Enhanced Distribution Alignment for Post-Training Quantization of Diffusion Modelscited by 23
  3. 2024A Comprehensive Survey of Convolutions in Deep Learning: Applications, Challenges, and Future Trendscited by 16
  4. 2024Self-calibration for Language Model Quantization and Pruningcited by 13
  5. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  6. 2025Zero-shot Quantization: A Comprehensive Surveycited by 8
  7. 2025Sharpness-Aware Data Generation for Zero-shot Quantizationcited by 5
  8. 2023Robustness-Guided Image Synthesis for Data-Free Quantizationcited by 4
  9. 2024Semantic Alignment and Reinforcement for Data-Free Quantization of Vision Transformerscited by 3
  10. 2025Enhancing Quantization-Aware Training on Edge Devices via Relative Entropy Coreset Selection and Cascaded Layer Correctioncited by 2
  11. 2023Dynamic Shuffle: An Efficient Channel Mixture Methodcited by 1
  12. 2024Infrared Domain Adaptation with Zero-Shot Quantizationcited by 1
3 more
  1. 2024Advanced Knowledge Transfer: Refined Feature Distillation for Zero-Shot Quantization in Edge Computingcited by 1
  2. 2026SynQ: Accurate Zero-shot Quantization by Synthesis-aware Fine-tuningcited by 1
  3. 2026Close Shortcut Wins Long: Seeking Diverse and Stable Generators for Data-Free Knowledge Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021A Generalized Zero-Shot Quantization of Deep Convolutional Neural Networks via Learned Weights Statistics
  2. 2022Long-Range Zero-Shot Generative Deep Network Quantization
  3. 2022Genie: Show Me the Data for Quantization
  4. 2025GranQ: Efficient Channel-wise Quantization via Vectorized Pre-Scaling for Zero-Shot QAT
  5. 2024Data Generation for Hardware-Friendly Post-Training Quantization
  6. 2023Zero-Shot Sharpness-Aware Quantization for Pre-trained Language Models
  7. 2025ZOQO: Zero-Order Quantized Optimization
  8. 2025Task-Specific Zero-shot Quantization-Aware Training for Object Detection