# TinyGAN: Distilling BigGAN for Conditional Image Generation

arXiv: https://arxiv.org/abs/2009.13829 · HTML: https://mlsanity.com/paper/2009.13829/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Sep 2020.

## Built on by (most cited first, 14 of 14)

- [StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets](https://mlsanity.com/paper/2202.00273/) (2022)
- [Content-Aware GAN Compression](https://mlsanity.com/paper/2104.02244/) (2021)
- [Online Multi-Granularity Distillation for GAN Compression](https://mlsanity.com/paper/2108.06908/) (2021)
- [MobileStyleGAN: A Lightweight Convolutional Neural Network for High-Fidelity Image Synthesis](https://mlsanity.com/paper/2104.04767/) (2021)
- [Mind the Gap in Distilling StyleGANs](https://mlsanity.com/paper/2208.08840/) (2022)
- [SAC-GAN: Structure-Aware Image Composition](https://mlsanity.com/paper/2112.06596/) (2021)
- [Microdosing: Knowledge Distillation for GAN based Compression](https://mlsanity.com/paper/2201.02624/) (2022)
- [SKDCGN: Source-free Knowledge Distillation of Counterfactual Generative Networks using cGANs](https://mlsanity.com/paper/2208.04226/) (2022)
- [On quantifying and improving realism of images generated with diffusion](https://mlsanity.com/paper/2309.14756/) (2023)
- [Compressing Image-to-Image Translation GANs Using Local Density Structures on Their Learned Manifold](https://mlsanity.com/paper/2312.14776/) (2023)
- [PPCD-GAN: Progressive Pruning and Class-Aware Distillation for Large-Scale Conditional GANs Compression](https://mlsanity.com/paper/2203.08456/) (2022)
- [CoroNetGAN: Controlled Pruning of GANs via Hypernetworks](https://mlsanity.com/paper/2403.08261/) (2024)
- [ZK-WAGON: Imperceptible Watermark for Image Generation Models using ZK-SNARKs](https://mlsanity.com/paper/2510.01967/) (2025)
- [Post-training Model Quantization Using GANs for Synthetic Data Generation](https://mlsanity.com/paper/2305.06052/) (2023)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) have become a powerful approach for generative image modeling. However, GANs are notorious for their training instability, especially on large-scale, complex datasets. While the recent work of BigGAN has significantly improved the quality of image generation on ImageNet, it requires a huge model, making it hard to deploy on resource-constrained devices. To reduce the model size, we propose a black-box knowledge distillation framework for compressing GANs, which highlights a stable and efficient training process. Given BigGAN as the teacher network, we manage to train a much smaller student network to mimic its functionality, achieving competitive performance on Inception and FID scores with the generator having 16× fewer parameters.