mlsanityData to 7 Oct 2026

Data-Free Quantization via Mixed-Precision Compensation without Fine-Tuning

arXiv 2307.00498 ↗First posted 2 Jul 2023cs.LG cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Neural network quantization is a very promising solution in the field of model compression, but its resulting accuracy highly depends on a training/fine-tuning process and requires the original data. This not only brings heavy computation and time costs but also is not conducive to privacy and sensitive information protection. Therefore, a few recent works are starting to focus on data-free quantization. However, data-free quantization does not perform well while dealing with ultra-low precision quantization. Although researchers utilize generative methods of synthetic data to address this problem partially, data synthesis needs to take a lot of computation and time. In this paper, we propose a data-free mixed-precision compensation (DF-MPC) method to recover the performance of an ultra-low precision quantized model without any data and fine-tuning process. By assuming the quantized error caused by a low-precision quantized layer can be restored via the reconstruction of a high-precision quantized layer, we mathematically formulate the reconstruction loss between the pre-trained full-precision model and its layer-wise mixed-precision quantized model. Based on our formulation, we theoretically deduce the closed-form solution by minimizing the reconstruction loss of the feature maps. Since DF-MPC does not require any original/synthetic data, it is a more efficient method to approximate the full-precision model. Experimentally, our DF-MPC is able to achieve higher accuracy for an ultra-low precision quantized model compared to the recent methods without any data and fine-tuning process.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 34 of its 43 references that are in the map. First posted on arXiv 2 Jul 2023.

Builds on

34 of 43 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  5. 2016Densely Connected Convolutional Networkscited by 6,691
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  8. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  9. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  10. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  11. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  12. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
22 more
  1. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  2. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  3. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  4. 2019Correlation Congruence for Knowledge Distillationcited by 280
  5. 2021Post-Training Quantization for Vision Transformercited by 274
  6. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  7. 2016Trained Ternary Quantizationcited by 203
  8. 2016Ternary Weight Networkscited by 195
  9. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  10. 2015Compressing Neural Networks with the Hashing Trickcited by 150
  11. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 150
  12. 2015Data-free parameter pruning for Deep Neural Networkscited by 112
  13. 2020Neural Pruning via Growing Regularizationcited by 97
  14. 2020Generative Low-bitwidth Data Free Quantizationcited by 87
  15. 2021Diversifying Sample Generation for Accurate Data-Free Quantizationcited by 63
  16. 2021Zero-shot Adversarial Quantizationcited by 44
  17. 2021Qimera: Data-free Quantization with Synthetic Boundary Supporting Samplescited by 39
  18. 2020Neuron Merging: Compensating for Pruned Neuronscited by 24
  19. 2020Computation-Efficient Knowledge Distillation via Uncertainty-Aware Mixupcited by 22
  20. 2021Generative Zero-shot Network Quantizationcited by 15
  21. 2020A Learning Framework for n-bit Quantized Neural Networks toward FPGAscited by 11
  22. 2023Learning Discretized Neural Networks under Ricci Flowcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 3; 2026: 1.

9 papers.

  1. 2024A Survey on Transformer Compressioncited by 44
  2. 2023Unified Data-Free Compression: Pruning and Quantization without Fine-Tuningcited by 22
  3. 2023Towards Trustworthy Dataset Distillationcited by 14
  4. 2024Timestep-Aware Correction for Quantized Diffusion Modelscited by 13
  5. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  6. 2025ConMamba: Contrastive Vision Mamba for Plant Disease Detectioncited by 2
  7. 2024AutoDFP: Automatic Data-Free Pruning via Channel Similarity Reconstructioncited by 1
  8. 2025Hyperbolic Binary Neural Networkcited by 0
  9. 2026Theoretical Guarantees for One-Shot Magnitude Pruning and Compute-Adaptive Early Exitcited by 0