# Convergence and Sample Complexity of SGD in GANs

arXiv: https://arxiv.org/abs/2012.00732 · HTML: https://mlsanity.com/paper/2012.00732/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2020.

## Built on by (most cited first, 2 of 2)

- [Variational inference with a quantum computer](https://mlsanity.com/paper/2103.06720/) (2021)
- [A Variance-Reduced Stochastic Accelerated Primal Dual Algorithm](https://mlsanity.com/paper/2202.09688/) (2022)

## Abstract (arXiv)

We provide theoretical convergence guarantees on training Generative Adversarial Networks (GANs) via SGD. We consider learning a target distribution modeled by a 1-layer Generator network with a non-linear activation function φ(·) parametrized by a d × d weight matrix mathbf W_*, i.e., f_*(mathbf x) = φ(mathbf W_* mathbf x). Our main result is that by training the Generator together with a Discriminator according to the Stochastic Gradient Descent-Ascent iteration proposed by Goodfellow et al. yields a Generator distribution that approaches the target distribution of f_*. Specifically, we can learn the target distribution within total-variation distance ε using tilde O(d2/ε2) samples which is (near-)information theoretically optimal. Our results apply to a broad class of non-linear activation functions φ, including ReLUs and is enabled by a connection with truncated statistics and an appropriate design of the Discriminator network. Our approach relies on a bilevel optimization framework to show that vanilla SGDA works.