# Distilling portable Generative Adversarial Networks for Image Translation

arXiv: https://arxiv.org/abs/2003.03519 · HTML: https://mlsanity.com/paper/2003.03519/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2020.

## Built on by (most cited first, 30 of 37)

- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillation](https://mlsanity.com/paper/2310.19444/) (2023)
- [Wavelet Knowledge Distillation: Towards Efficient Image-to-Image Translation](https://mlsanity.com/paper/2203.06321/) (2022)
- [Teachers Do More Than Teach: Compressing Image-to-Image Models](https://mlsanity.com/paper/2103.03467/) (2021)
- [Undistillable: Making A Nasty Teacher That CANNOT teach students](https://mlsanity.com/paper/2105.07381/) (2021)
- [Content-Aware GAN Compression](https://mlsanity.com/paper/2104.02244/) (2021)
- [Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective](https://mlsanity.com/paper/2103.00397/) (2021)
- [Online Multi-Granularity Distillation for GAN Compression](https://mlsanity.com/paper/2108.06908/) (2021)
- [Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme](https://mlsanity.com/paper/2110.14439/) (2021)
- [Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillation](https://mlsanity.com/paper/2306.10687/) (2023)
- [MFAGAN: A Compression Framework for Memory-Efficient On-Device Super-Resolution GAN](https://mlsanity.com/paper/2107.12679/) (2021)
- [Fast-Vid2Vid: Spatial-Temporal Compression for Video-to-Video Synthesis](https://mlsanity.com/paper/2207.05049/) (2022)
- [Fast and High-Performance Learned Image Compression With Improved Checkerboard Context Model, Deformable Residual Module, and Knowledge Distillation](https://mlsanity.com/paper/2309.02529/) (2023)
- [Mind the Gap in Distilling StyleGANs](https://mlsanity.com/paper/2208.08840/) (2022)
- [Calibrating a Deep Neural Network with Its Predecessors](https://mlsanity.com/paper/2302.06245/) (2023)
- [GhostSR: Learning Ghost Features for Efficient Image Super-Resolution](https://mlsanity.com/paper/2101.08525/) (2021)
- [Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity](https://mlsanity.com/paper/2102.11382/) (2021)
- [Don't Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance](https://mlsanity.com/paper/2203.02770/) (2022)
- [MineGAN++: Mining Generative Models for Efficient Knowledge Transfer to Limited Data Domains](https://mlsanity.com/paper/2104.13742/) (2021)
- [Hypercomplex Image-to-Image Translation](https://mlsanity.com/paper/2205.02087/) (2022)
- [Discriminator-Cooperated Feature Map Distillation for GAN Compression](https://mlsanity.com/paper/2212.14169/) (2022)
- [Coarse-to-Fine Searching for Efficient Generative Adversarial Networks](https://mlsanity.com/paper/2104.09223/) (2021)
- [Structured Knowledge Distillation Towards Efficient and Compact Multi-View 3D Detection](https://mlsanity.com/paper/2211.08398/) (2022)
- [Head3D: Complete 3D Head Generation via Tri-plane Feature Distillation](https://mlsanity.com/paper/2303.15892/) (2023)
- [Compressing Image-to-Image Translation GANs Using Local Density Structures on Their Learned Manifold](https://mlsanity.com/paper/2312.14776/) (2023)
- [Anti-Compression Contrastive Facial Forgery Detection](https://mlsanity.com/paper/2302.06183/) (2023)
- [CoroNetGAN: Controlled Pruning of GANs via Hypernetworks](https://mlsanity.com/paper/2403.08261/) (2024)
- [Soft Knowledge Distillation with Multi-Dimensional Cross-Net Attention for Image Restoration Models Compression](https://mlsanity.com/paper/2501.09321/) (2025)
- [Evaluating Post-Training Compression in GANs using Locality-Sensitive Hashing](https://mlsanity.com/paper/2103.11912/) (2021)
- [ChildPredictor: A Child Face Prediction Framework with Disentangled Learning](https://mlsanity.com/paper/2204.09962/) (2022)

## Abstract (arXiv)

Despite Generative Adversarial Networks (GANs) have been widely used in various image-to-image translation tasks, they can be hardly applied on mobile devices due to their heavy computation and storage cost. Traditional network compression methods focus on visually recognition tasks, but never deal with generation tasks. Inspired by knowledge distillation, a student generator of fewer parameters is trained by inheriting the low-level and high-level information from the original heavy teacher generator. To promote the capability of student generator, we include a student discriminator to measure the distances between real images, and images generated by student and teacher generators. An adversarial learning process is therefore established to optimize student generator and student discriminator. Qualitative and quantitative analysis by conducting experiments on benchmark datasets demonstrate that the proposed method can learn portable generative models with strong performance.