mlsanityData to 7 Oct 2026

Dream Distillation: A Data-Independent Model Compression Framework

arXiv 1905.07072 ↗First posted 17 May 2019stat.ML cs.CV cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

Model compression is eminently suited for deploying deep learning on IoT-devices. However, existing model compression techniques rely on access to the original or some alternate dataset. In this paper, we address the model compression problem when no real data is available, e.g., when data is private. To this end, we propose Dream Distillation, a data-independent model compression framework. Our experiments show that Dream Distillation can achieve 88.5% accuracy on the CIFAR-10 test set without actually training on the original data!

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 4; 2023: 2; 2024: 6; 2025: 1.

19 papers.

Sort:

By area:

  1. 2022DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformerscited by 65
  2. 2021Data-Free Knowledge Transfer: A Surveycited by 31
  3. 2023PaD: Program-aided Distillation Can Teach Small Models Reasoning Better than Chain-of-thought Fine-tuningcited by 29
  4. 2024VkD: Improving Knowledge Distillation using Orthogonal Projectionscited by 24
  5. 2024Small Scale Data-Free Knowledge Distillationcited by 20
  6. 2022IDEAL: Query-Efficient Data-Free Learning from Black-box Modelscited by 16
  7. 2021Generative Zero-shot Network Quantizationcited by 15
  8. 2024De-confounded Data-free Knowledge Distillation for Handling Distribution Shiftscited by 11
  9. 2021Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentationcited by 8
  10. 2021Enhancing Data-Free Adversarial Distillation with Activation Regularization and Virtual Interpolationcited by 7
  11. 2023Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillationcited by 7
  12. 2024Relation-Guided Adversarial Learning for Data-free Knowledge Transfercited by 6
7 more
  1. 2021Representation Consolidation for Training Expert Studentscited by 5
  2. 2022Scalable Collaborative Learning via Representation Sharingcited by 3
  3. 2021Super-Efficient Super Resolution for Fast Adversarial Defense at the Edgecited by 2
  4. 2024Data-free Distillation with Degradation-prompt Diffusion for Multi-weather Image Restorationcited by 2
  5. 2022Explaining Neural Networks without Access to Training Datacited by 1
  6. 2025From Large to Super-Tiny: End-to-End Optimization for Cost-Efficient LLMscited by 1
  7. 2024Learning to Project for Cross-Task Knowledge Distillationcited by 0