# GAN Slimming: All-in-One GAN Compression by A Unified Optimization Framework

arXiv: https://arxiv.org/abs/2008.11062 · HTML: https://mlsanity.com/paper/2008.11062/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Aug 2020.

## Built on by (most cited first, 30 of 44)

- [A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendations](https://mlsanity.com/paper/2308.06767/) (2023)
- [TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up](https://mlsanity.com/paper/2102.07074/) (2021)
- [Anycost GANs for Interactive Image Synthesis and Editing](https://mlsanity.com/paper/2103.03243/) (2021)
- [Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models](https://mlsanity.com/paper/2211.02048/) (2022)
- [Teachers Do More Than Teach: Compressing Image-to-Image Models](https://mlsanity.com/paper/2103.03467/) (2021)
- [GANs Can Play Lottery Tickets Too](https://mlsanity.com/paper/2106.00134/) (2021)
- [GDP: Stabilized Neural Network Pruning via Gates with Differentiable Polarization](https://mlsanity.com/paper/2109.02220/) (2021)
- [Content-Aware GAN Compression](https://mlsanity.com/paper/2104.02244/) (2021)
- [Online Multi-Granularity Distillation for GAN Compression](https://mlsanity.com/paper/2108.06908/) (2021)
- [Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Scheme](https://mlsanity.com/paper/2110.14439/) (2021)
- [MFAGAN: A Compression Framework for Memory-Efficient On-Device Super-Resolution GAN](https://mlsanity.com/paper/2107.12679/) (2021)
- [Fast and Memory-Efficient Video Diffusion Using Streamlined Inference](https://mlsanity.com/paper/2411.01171/) (2024)
- [Fast-Vid2Vid: Spatial-Temporal Compression for Video-to-Video Synthesis](https://mlsanity.com/paper/2207.05049/) (2022)
- [MobileStyleGAN: A Lightweight Convolutional Neural Network for High-Fidelity Image Synthesis](https://mlsanity.com/paper/2104.04767/) (2021)
- [Region-aware Knowledge Distillation for Efficient Image-to-Image Translation](https://mlsanity.com/paper/2205.12451/) (2022)
- [Mind the Gap in Distilling StyleGANs](https://mlsanity.com/paper/2208.08840/) (2022)
- [Balanced Training for Sparse GANs](https://mlsanity.com/paper/2302.14670/) (2023)
- [Don't Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance](https://mlsanity.com/paper/2203.02770/) (2022)
- [Generative Optimization Networks for Memory Efficient Data Generation](https://mlsanity.com/paper/2110.02912/) (2021)
- [Cut Inner Layers: A Structured Pruning Strategy for Efficient U-Net GANs](https://mlsanity.com/paper/2206.14658/) (2022)
- [Diversity-aware Channel Pruning for StyleGAN Compression](https://mlsanity.com/paper/2403.13548/) (2024)
- [Quantization of Generative Adversarial Networks for Efficient Inference: a Methodological Study](https://mlsanity.com/paper/2108.13996/) (2021)
- [Can We Find Strong Lottery Tickets in Generative Models?](https://mlsanity.com/paper/2212.08311/) (2022)
- [Discriminator-Cooperated Feature Map Distillation for GAN Compression](https://mlsanity.com/paper/2212.14169/) (2022)
- [Coarse-to-Fine Searching for Efficient Generative Adversarial Networks](https://mlsanity.com/paper/2104.09223/) (2021)
- [Black-Box Diagnosis and Calibration on GAN Intra-Mode Collapse: A Pilot Study](https://mlsanity.com/paper/2107.12202/) (2021)
- [Head3D: Complete 3D Head Generation via Tri-plane Feature Distillation](https://mlsanity.com/paper/2303.15892/) (2023)
- [Compressing Image-to-Image Translation GANs Using Local Density Structures on Their Learned Manifold](https://mlsanity.com/paper/2312.14776/) (2023)
- [Singular Value Scaling: Efficient Generative Model Compression via Pruned Weights Refinement](https://mlsanity.com/paper/2412.17387/) (2024)
- [DRAGON: Decentralized Fault Tolerance in Edge Federations](https://mlsanity.com/paper/2208.07658/) (2022)

## Abstract (arXiv)

Generative adversarial networks (GANs) have gained increasing popularity in various computer vision applications, and recently start to be deployed to resource-constrained mobile devices. Similar to other deep models, state-of-the-art GANs suffer from high parameter complexities. That has recently motivated the exploration of compressing GANs (usually generators). Compared to the vast literature and prevailing success in compressing deep classifiers, the study of GAN compression remains in its infancy, so far leveraging individual compression techniques instead of more sophisticated combinations. We observe that due to the notorious instability of training GANs, heuristically stacking different compression techniques will result in unsatisfactory results. To this end, we propose the first unified optimization framework combining multiple compression means for GAN compression, dubbed GAN Slimming (GS). GS seamlessly integrates three mainstream compression techniques: model distillation, channel pruning and quantization, together with the GAN minimax objective, into one unified optimization form, that can be efficiently optimized from end to end. Without bells and whistles, GS largely outperforms existing options in compressing image-to-image translation GANs. Specifically, we apply GS to compress CartoonGAN, a state-of-the-art style transfer network, by up to 47 times, with minimal visual quality degradation. Codes and pre-trained models can be found at https://github.com/TAMU-VITA/GAN-Slimming.