# Top-k Training of GANs: Improving GAN Performance by Throwing Away Bad Samples

arXiv: https://arxiv.org/abs/2002.06224 · HTML: https://mlsanity.com/paper/2002.06224/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2020.

## Built on by (most cited first, 4 of 4)

- [SAN: Inducing Metrizability of GAN with Discriminative Normalized Linear Layer](https://mlsanity.com/paper/2301.12811/) (2023)
- [Enhancing Stability in Training Conditional Generative Adversarial Networks via Selective Data Matching](https://mlsanity.com/paper/2107.08792/) (2021)
- [Sinogram Enhancement with Generative Adversarial Networks using Shape Priors](https://mlsanity.com/paper/2202.00419/) (2022)
- [SONA: Learning Conditional, Unconditional, and Mismatching-Aware Discriminator](https://mlsanity.com/paper/2510.04576/) (2025)

## Abstract (arXiv)

We introduce a simple (one line of code) modification to the Generative Adversarial Network (GAN) training algorithm that materially improves results with no increase in computational cost: When updating the generator parameters, we simply zero out the gradient contributions from the elements of the batch that the critic scores as `least realistic'. Through experiments on many different GAN variants, we show that this `top-k update' procedure is a generally applicable improvement. In order to understand the nature of the improvement, we conduct extensive analysis on a simple mixture-of-Gaussians dataset and discover several interesting phenomena. Among these is that, when gradient updates are computed using the worst-scoring batch elements, samples can actually be pushed further away from their nearest mode. We also apply our method to recent GAN variants and improve state-of-the-art FID for conditional generation from 9.21 to 8.57 on CIFAR-10.