mlsanityData to 13 Oct 2026

Data-Free Knowledge Amalgamation via Group-Stack Dual-GAN

arXiv 2003.09088 ↗First posted 20 Mar 2020cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

Recent advances in deep learning have provided procedures for learning one network to amalgamate multiple streams of knowledge from the pre-trained Convolutional Neural Network (CNN) models, thus reduce the annotation cost. However, almost all existing methods demand massive training data, which may be unavailable due to privacy or transmission issues. In this paper, we propose a data-free knowledge amalgamate strategy to craft a well-behaved multi-task student network from multiple single/multi-task teachers. The main idea is to construct the group-stack generative adversarial networks (GANs) which have two dual generators. First one generator is trained to collect the knowledge by reconstructing the images approximating the original dataset utilized for pre-training the teachers. Then a dual generator is trained by taking the output from the former generator as input. Finally we treat the dual part generator as the target network and regroup it. As demonstrated on several benchmarks of multi-label classification, the proposed method without any training data achieves the surprisingly competitive results, even compared with some full-supervised methods.

Key facts. Cited by 32 papers in the map (data through 13 Oct 2026). First posted on arXiv 20 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 8; 2022: 9; 2023: 9; 2024: 4; 2025: 1; 2026: 1.

32 papers.

Sort:

By area:

  1. 2021Source-Free Domain Adaptation for Semantic Segmentationcited by 168
  2. 2021Contrastive Model Inversion for Data-Free Knowledge Distillationcited by 66
  3. 2022DearKD: Data-Efficient Early Knowledge Distillation for Vision Transformerscited by 65
  4. 2021Zero-shot Adversarial Quantizationcited by 44
  5. 2024Training-Free Pretrained Model Mergingcited by 32
  6. 2021Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Datacited by 31
  7. 2021Data-Free Knowledge Transfer: A Surveycited by 31
  8. 2022Learning with Recoverable Forgettingcited by 30
  9. 2023TinyMIM: An Empirical Study of Distilling MIM Pre-trained Modelscited by 24
  10. 2023Deep Graph Reprogrammingcited by 22
  11. 2022IDEAL: Query-Efficient Data-Free Learning from Black-box Modelscited by 16
  12. 2022A survey on GANs for computer vision: Recent research, analysis and taxonomycited by 15
20 more
  1. 2022CDFKD-MFS: Collaborative Data-free Knowledge Distillation via Multi-level Feature Sharingcited by 15
  2. 2023FoPro-KD: Fourier Prompted Effective Knowledge Distillation for Long-Tailed Medical Image Recognitioncited by 9
  3. 2023Revisiting Data-Free Knowledge Distillation with Poisoned Teacherscited by 9
  4. 2023Sampling to Distill: Knowledge Transfer from Open-World Datacited by 8
  5. 2021The Augmented Image Prior: Distilling 1000 Classes by Extrapolating from a Single Imagecited by 7
  6. 2023Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillationcited by 7
  7. 2021Training Generative Adversarial Networks in One Stagecited by 6
  8. 2023Partial Network Cloningcited by 6
  9. 2021Representation Consolidation for Training Expert Studentscited by 5
  10. 2022Knowledge Amalgamation for Object Detection with Transformerscited by 5
  11. 2023Synthetic data generation method for data-free knowledge distillation in regression neural networkscited by 4
  12. 2022Aligning Logits Generatively for Principled Black-Box Knowledge Distillationcited by 3
  13. 2024Encapsulating Knowledge in One Promptcited by 3
  14. 2022Optimizer Amalgamationcited by 1
  15. 2024Diffusion-based Data Augmentation and Knowledge Distillation with Generated Soft Labels Solving Data Scarcity Problems of SAR Oil Spill Segmentationcited by 1
  16. 2025Distilling A Universal Expert from Clustered Federated Learningcited by 1
  17. 2022Self-distilled Knowledge Delegator for Exemplar-free Class Incremental Learningcited by 0
  18. 2023Tackling Intertwined Data and Device Heterogeneities in Federated Learning with Unlimited Stalenesscited by 0
  19. 2024Information Extraction from Heterogeneous Documents without Ground Truth Labels using Synthetic Label Generation and Knowledge Distillationcited by 0
  20. 2026IDeaL: Data-Free Multi-Teacher Distillation via Improved Dead Leavescited by 0