# DeGAN : Data-Enriching GAN for Retrieving Representative Samples from a Trained Classifier

arXiv: https://arxiv.org/abs/1912.11960 · HTML: https://mlsanity.com/paper/1912.11960/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Dec 2019.

## Built on by (most cited first, 21 of 21)

- [Domain Impression: A Source Data Free Domain Adaptation Method](https://mlsanity.com/paper/2102.09003/) (2021)
- [Towards Data-Free Model Stealing in a Hard Label Setting](https://mlsanity.com/paper/2204.11022/) (2022)
- [DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memories](https://mlsanity.com/paper/2111.04625/) (2021)
- [Robust and Resource-Efficient Data-Free Knowledge Distillation by Generative Pseudo Replay](https://mlsanity.com/paper/2201.03019/) (2022)
- [Lion: Adversarial Distillation of Proprietary Large Language Models](https://mlsanity.com/paper/2305.12870/) (2023)
- [Preventing Catastrophic Forgetting and Distribution Mismatch in Knowledge Distillation via Synthetic Data](https://mlsanity.com/paper/2108.05698/) (2021)
- [Data-Free Knowledge Transfer: A Survey](https://mlsanity.com/paper/2112.15278/) (2021)
- [Learning to Retain while Acquiring: Combating Distribution-Shift in Adversarial Data-Free Knowledge Distillation](https://mlsanity.com/paper/2302.14290/) (2023)
- [Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging](https://mlsanity.com/paper/2107.09543/) (2021)
- [Mining Data Impressions from Deep Models as Substitute for the Unavailable Training Data](https://mlsanity.com/paper/2101.06069/) (2021)
- [Representation Consolidation for Training Expert Students](https://mlsanity.com/paper/2107.08039/) (2021)
- [Scalable Collaborative Learning via Representation Sharing](https://mlsanity.com/paper/2211.10943/) (2022)
- [Encapsulating Knowledge in One Prompt](https://mlsanity.com/paper/2407.11902/) (2024)
- [Towards Few-Call Model Stealing via Active Self-Paced Knowledge Distillation and Diffusion-Based Image Generation](https://mlsanity.com/paper/2310.00096/) (2023)
- [Mosaic: Data-Free Knowledge Distillation via Mixture-of-Experts for Heterogeneous Distributed Environments](https://mlsanity.com/paper/2505.19699/) (2025)
- [Efficient Model Extraction via Boundary Sampling](https://mlsanity.com/paper/2410.15429/) (2024)
- [Self-distilled Knowledge Delegator for Exemplar-free Class Incremental Learning](https://mlsanity.com/paper/2205.11071/) (2022)
- [Tackling Intertwined Data and Device Heterogeneities in Federated Learning with Unlimited Staleness](https://mlsanity.com/paper/2309.13536/) (2023)
- [CaBaGe: Data-Free Model Extraction using ClAss BAlanced Generator Ensemble](https://mlsanity.com/paper/2409.10643/) (2024)
- [FedRecon: Missing Modality Reconstruction in Heterogeneous Distributed Environments](https://mlsanity.com/paper/2504.09941/) (2025)
- [Improving Diversity in Black-box Few-shot Knowledge Distillation](https://mlsanity.com/paper/2604.25795/) (2026)

## Abstract (arXiv)

In this era of digital information explosion, an abundance of data from numerous modalities is being generated as well as archived everyday. However, most problems associated with training Deep Neural Networks still revolve around lack of data that is rich enough for a given task. Data is required not only for training an initial model, but also for future learning tasks such as Model Compression and Incremental Learning. A diverse dataset may be used for training an initial model, but it may not be feasible to store it throughout the product life cycle due to data privacy issues or memory constraints. We propose to bridge the gap between the abundance of available data and lack of relevant data, for the future learning tasks of a given trained network. We use the available data, that may be an imbalanced subset of the original training dataset, or a related domain dataset, to retrieve representative samples from a trained classifier, using a novel Data-enriching GAN (DeGAN) framework. We demonstrate that data from a related domain can be leveraged to achieve state-of-the-art performance for the tasks of Data-free Knowledge Distillation and Incremental Learning on benchmark datasets. We further demonstrate that our proposed framework can enrich any data, even from unrelated domains, to make it more useful for the future learning tasks of a given network.