# AlphaGAN: Fully Differentiable Architecture Search for Generative Adversarial Networks

arXiv: https://arxiv.org/abs/2006.09134 · HTML: https://mlsanity.com/paper/2006.09134/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2020.

## Built on by (most cited first, 10 of 10)

- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse](https://mlsanity.com/paper/2112.04660/) (2021)
- [Local Stochastic Bilevel Optimization with Momentum-Based Variance Reduction](https://mlsanity.com/paper/2205.01608/) (2022)
- [EGANS: Evolutionary Generative Adversarial Network Search for Zero-Shot Learning](https://mlsanity.com/paper/2308.09915/) (2023)
- [EAGAN: Efficient Two-stage Evolutionary Architecture Search for GANs](https://mlsanity.com/paper/2111.15097/) (2021)
- [Differentiable Bilevel Programming for Stackelberg Congestion Games](https://mlsanity.com/paper/2209.07618/) (2022)
- [Manifold-preserved GANs](https://mlsanity.com/paper/2109.08955/) (2021)
- [DGL-GAN: Discriminator Guided Learning for GAN Compression](https://mlsanity.com/paper/2112.06502/) (2021)
- [SNED: Superposition Network Architecture Search for Efficient Video Diffusion Model](https://mlsanity.com/paper/2406.00195/) (2024)
- [Neural Architecture Search for Generative Adversarial Networks: A Comprehensive Review and Critical Analysis](https://mlsanity.com/paper/2606.26169/) (2026)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) are formulated as minimax game problems, whereby generators attempt to approach real data distributions by virtue of adversarial learning against discriminators. The intrinsic problem complexity poses the challenge to enhance the performance of generative networks. In this work, we aim to boost model learning from the perspective of network architectures, by incorporating recent progress on automated architecture search into GANs. To this end, we propose a fully differentiable search framework for generative adversarial networks, dubbed alphaGAN. The searching process is formalized as solving a bi-level minimax optimization problem, in which the outer-level objective aims for seeking a suitable network architecture towards pure Nash Equilibrium conditioned on the generator and the discriminator network parameters optimized with a traditional GAN loss in the inner level. The entire optimization performs a first-order method by alternately minimizing the two-level objective in a fully differentiable manner, enabling architecture search to be completed in an enormous search space. Extensive experiments on CIFAR-10 and STL-10 datasets show that our algorithm can obtain high-performing architectures only with 3-GPU hours on a single GPU in the search space comprised of approximate 2 ? 1011 possible configurations. We also provide a comprehensive analysis on the behavior of the searching process and the properties of searched architectures, which would benefit further research on architectures for generative models. Pretrained models and codes are available at https://github.com/yuesongtian/AlphaGAN.