mlsanityData to 7 Oct 2026

Discriminator-Cooperated Feature Map Distillation for GAN Compression

arXiv 2212.14169 ↗First posted 29 Dec 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Despite excellent performance in image generation, Generative Adversarial Networks (GANs) are notorious for its requirements of enormous storage and intensive computation. As an awesome ''performance maker'', knowledge distillation is demonstrated to be particularly efficacious in exploring low-priced GANs. In this paper, we investigate the irreplaceability of teacher discriminator and present an inventive discriminator-cooperated distillation, abbreviated as DCD, towards refining better feature maps from the generator. In contrast to conventional pixel-to-pixel match methods in feature map distillation, our DCD utilizes teacher discriminator as a transformation to drive intermediate results of the student generator to be perceptually close to corresponding outputs of the teacher generator. Furthermore, in order to mitigate mode collapse in GAN compression, we construct a collaborative adversarial training paradigm where the teacher discriminator is from scratch established to co-train with student generator in company with our DCD. Our DCD shows superior results compared with existing GAN compression methods. For instance, after reducing over 40x MACs and 80x parameters of CycleGAN, we well decrease FID metric from 61.53 to 48.24 while the current SoTA method merely has 51.92. This work's source code has been made accessible at https://github.com/poopit/DCD-official.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 43 of its 51 references that are in the map. First posted on arXiv 29 Dec 2022.

Builds on

43 of 51 references.

References · 43

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  4. 2022Generative Adversarial Networkscited by 5,603
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  8. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  11. 2014FitNets: Hints for Thin Deep Netscited by 1,619
  12. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
31 more
  1. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  2. 2019Relational Knowledge Distillationcited by 873
  3. 2018Self-Attention Generative Adversarial Networkscited by 809
  4. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  5. 2019Similarity-Preserving Knowledge Distillationcited by 507
  6. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  7. 2022Decoupled Knowledge Distillationcited by 380
  8. 2019A Comprehensive Overhaul of Feature Distillationcited by 341
  9. 2021Distilling Knowledge via Knowledge Reviewcited by 285
  10. 2022Masked Generative Distillationcited by 107
  11. 2020Wasserstein Contrastive Representation Distillationcited by 63
  12. 2021Anycost GANs for Interactive Image Synthesis and Editingcited by 54
  13. 2020AutoGAN-Distiller: Searching to Compress Generative Adversarial Networkscited by 50
  14. 2020GAN Slimming: All-in-One GAN Compression by A Unified Optimization Frameworkcited by 44
  15. 2022Exploring Patch-wise Semantic Relation for Contrastive Learning in Image-to-Image Translation Taskscited by 42
  16. 2022Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translationcited by 39
  17. 2021DRB-GAN: A Dynamic ResBlock Generative Adversarial Network for Artistic Style Transfercited by 38
  18. 2021Teachers Do More Than Teach: Compressing Image-to-Image Modelscited by 37
  19. 2020Distilling portable Generative Adversarial Networks for Image Translationcited by 36
  20. 2019Co-Evolutionary Compression for Unpaired Image Translationcited by 32
  21. 2021GANs Can Play Lottery Tickets Toocited by 32
  22. 2019Compressing GANs using Knowledge Distillationcited by 31
  23. 2020Slimmable Generative Adversarial Networkscited by 27
  24. 2021Online Multi-Granularity Distillation for GAN Compressioncited by 24
  25. 2021Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Schemecited by 23
  26. 2019QGAN: Quantized Generative Adversarial Networkscited by 20
  27. 2022Knowledge Condensation Distillationcited by 20
  28. 2020Learning Efficient GANs for Image Translation via Differentiable Masks and co-Attention Distillationcited by 17
  29. 2021Semantic Relation Preserving Knowledge Distillation for Image-to-Image Translationcited by 13
  30. 2022Cut Inner Layers: A Structured Pruning Strategy for Efficient U-Net GANscited by 5
  31. 2022Exploring Content Relationships for Distilling Efficient GANscited by 1

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024Diversity-aware Channel Pruning for StyleGAN Compressioncited by 5
  2. 2024Nickel and Diming Your GAN: A Dual-Method Approach to Enhancing GAN Efficiency via Knowledge Distillationcited by 2
  3. 2025Leveraging Modified Ex Situ Tomography Data for Segmentation of In Situ Synchrotron X-Ray Computed Tomographycited by 1
  4. 2025SDAKD: Student Discriminator Assisted Knowledge Distillation for Super-Resolution Generative Adversarial Networkscited by 0