# MixMix: All You Need for Data-Free Compression Are Feature and Data Mixing

arXiv: https://arxiv.org/abs/2011.09899 · HTML: https://mlsanity.com/paper/2011.09899/

## Key facts

- Cited by 29 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 19 Nov 2020.

## Built on by (most cited first, 29 of 29)

- [QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantization](https://mlsanity.com/paper/2203.05740/) (2022)
- [Neuromorphic Data Augmentation for Training Spiking Neural Networks](https://mlsanity.com/paper/2203.06145/) (2022)
- [Revisiting Random Channel Pruning for Neural Network Compression](https://mlsanity.com/paper/2205.05676/) (2022)
- [MQBench: Towards Reproducible and Deployable Model Quantization Benchmark](https://mlsanity.com/paper/2111.03759/) (2021)
- [Data-Free Knowledge Transfer: A Survey](https://mlsanity.com/paper/2112.15278/) (2021)
- [Small Scale Data-Free Knowledge Distillation](https://mlsanity.com/paper/2406.07876/) (2024)
- [Genie: Show Me the Data for Quantization](https://mlsanity.com/paper/2212.04780/) (2022)
- [Hard Sample Matters a Lot in Zero-Shot Quantization](https://mlsanity.com/paper/2303.13826/) (2023)
- [A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services](https://mlsanity.com/paper/2304.14271/) (2023)
- [Towards Data-Free Domain Generalization](https://mlsanity.com/paper/2110.04545/) (2021)
- [Fine-grained Data Distribution Alignment for Post-Training Quantization](https://mlsanity.com/paper/2109.04186/) (2021)
- [Practical Network Acceleration with Tiny Sets](https://mlsanity.com/paper/2202.07861/) (2022)
- [Privacy Vulnerability of Split Computing to Data-Free Model Inversion Attacks](https://mlsanity.com/paper/2107.06304/) (2021)
- [GenQ: Quantization in Low Data Regimes with Generative Synthetic Data](https://mlsanity.com/paper/2312.05272/) (2023)
- [Zero-shot Quantization: A Comprehensive Survey](https://mlsanity.com/paper/2505.09188/) (2025)
- [REx: Data-Free Residual Quantization Error Expansion](https://mlsanity.com/paper/2203.14645/) (2022)
- [Converting Artificial Neural Networks to Spiking Neural Networks via Parameter Calibration](https://mlsanity.com/paper/2205.10121/) (2022)
- [Knowledge Distillation Detection for Open-weights Models](https://mlsanity.com/paper/2510.02302/) (2025)
- [FedD2S: Personalized Data-Free Federated Knowledge Distillation](https://mlsanity.com/paper/2402.10846/) (2024)
- [Beyond Classification: Knowledge Distillation using Multi-Object Impressions](https://mlsanity.com/paper/2110.14215/) (2021)
- [TAGPerson: A Target-Aware Generation Pipeline for Person Re-identification](https://mlsanity.com/paper/2112.14239/) (2021)
- [ACQ: Improving Generative Data-free Quantization Via Attention Correction](https://mlsanity.com/paper/2301.07266/) (2023)
- [Practical Network Acceleration with Tiny Sets: Hypothesis, Theory, and Algorithm](https://mlsanity.com/paper/2303.00972/) (2023)
- [Enhancing Generalization in Data-free Quantization via Mixup-class Prompting](https://mlsanity.com/paper/2507.21947/) (2025)
- [PRISM: Precision-Recall Informed Data-Free Knowledge Distillation via Generative Diffusion](https://mlsanity.com/paper/2509.16897/) (2025)
- [Data-Free Backbone Fine-Tuning for Pruned Neural Networks](https://mlsanity.com/paper/2306.12881/) (2023)
- [Gradient-Based Post-Training Quantization: Challenging the Status Quo](https://mlsanity.com/paper/2308.07662/) (2023)
- [PIPE : Parallelized Inference Through Post-Training Quantization Ensembling of Residual Expansions](https://mlsanity.com/paper/2311.15806/) (2023)
- [GranQ: Efficient Channel-wise Quantization via Vectorized Pre-Scaling for Zero-Shot QAT](https://mlsanity.com/paper/2503.18339/) (2025)

## Abstract (arXiv)

User data confidentiality protection is becoming a rising challenge in the present deep learning research. Without access to data, conventional data-driven model compression faces a higher risk of performance degradation. Recently, some works propose to generate images from a specific pretrained model to serve as training data. However, the inversion process only utilizes biased feature statistics stored in one model and is from low-dimension to high-dimension. As a consequence, it inevitably encounters the difficulties of generalizability and inexact inversion, which leads to unsatisfactory performance. To address these problems, we propose MixMix based on two simple yet effective techniques: (1) Feature Mixing: utilizes various models to construct a universal feature space for generalized inversion; (2) Data Mixing: mixes the synthesized images and labels to generate exact label information. We prove the effectiveness of MixMix from both theoretical and empirical perspectives. Extensive experiments show that MixMix outperforms existing methods on the mainstream compression tasks, including quantization, knowledge distillation, and pruning. Specifically, MixMix achieves up to 4% and 20% accuracy uplift on quantization and pruning, respectively, compared to existing data-free compression work.