# Small-GAN: Speeding Up GAN Training Using Core-sets

arXiv: https://arxiv.org/abs/1910.13540 · HTML: https://mlsanity.com/paper/1910.13540/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2019.

## Built on by (most cited first, 30 of 41)

- [Towards Total Recall in Industrial Anomaly Detection](https://mlsanity.com/paper/2106.08265/) (2021)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning](https://mlsanity.com/paper/2204.08499/) (2022)
- [Towards Faster and Stabilized GAN Training for High-fidelity Few-shot Image Synthesis](https://mlsanity.com/paper/2101.04775/) (2021)
- [Dataset Quantization](https://mlsanity.com/paper/2308.10524/) (2023)
- [ReConPatch : Contrastive Patch Representation Learning for Industrial Anomaly Detection](https://mlsanity.com/paper/2305.16713/) (2023)
- [Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models](https://mlsanity.com/paper/2408.02085/) (2024)
- [A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advances](https://mlsanity.com/paper/2505.17799/) (2025)
- [Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective](https://mlsanity.com/paper/2103.00397/) (2021)
- [Core-set Sampling for Efficient Neural Architecture Search](https://mlsanity.com/paper/2107.06869/) (2021)
- [SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning](https://mlsanity.com/paper/2405.00705/) (2024)
- [A Unified Approach to Coreset Learning](https://mlsanity.com/paper/2111.03044/) (2021)
- [FAPM: Fast Adaptive Patch Memory for Real-time Industrial Anomaly Detection](https://mlsanity.com/paper/2211.07381/) (2022)
- [DOS: Diverse Outlier Sampling for Out-of-Distribution Detection](https://mlsanity.com/paper/2306.02031/) (2023)
- [Accelerating Neural Architecture Search via Proxy Data](https://mlsanity.com/paper/2106.04784/) (2021)
- [Adversarial Robustness of Streaming Algorithms through Importance Sampling](https://mlsanity.com/paper/2106.14952/) (2021)
- [Dataset Quantization with Active Learning based Adaptive Sampling](https://mlsanity.com/paper/2407.07268/) (2024)
- [Data pruning and neural scaling laws: fundamental limitations of score-based algorithms](https://mlsanity.com/paper/2302.06960/) (2023)
- [Beyond Efficiency: Molecular Data Pruning for Enhanced Generalization](https://mlsanity.com/paper/2409.01081/) (2024)
- [Generative AI Enables Medical Image Segmentation in Ultra Low-Data Regimes](https://mlsanity.com/paper/2408.17421/) (2024)
- [Towards Zero-shot 3D Anomaly Localization](https://mlsanity.com/paper/2412.04304/) (2024)
- [Obstacle Aware Sampling for Path Planning](https://mlsanity.com/paper/2203.04075/) (2022)
- [Model Performance-Guided Evaluation Data Selection for Effective Prompt Optimization](https://mlsanity.com/paper/2505.10736/) (2025)
- [Dataset Condensation with Color Compensation](https://mlsanity.com/paper/2508.01139/) (2025)
- [One Backward from Ten Forward, Subsampling for Large-Scale Deep Learning](https://mlsanity.com/paper/2104.13114/) (2021)
- [ALGAN: Anomaly Detection by Generating Pseudo Anomalous Data via Latent Variables](https://mlsanity.com/paper/2202.10281/) (2022)
- [ATOM: Attention Mixer for Efficient Dataset Distillation](https://mlsanity.com/paper/2405.01373/) (2024)
- [Leverage Score Sampling for Complete Mode Coverage in Generative Adversarial Networks](https://mlsanity.com/paper/2104.02373/) (2021)
- [FMGNN: Fused Manifold Graph Neural Network](https://mlsanity.com/paper/2304.01081/) (2023)
- [Leveraging Open Knowledge for Advancing Task Expertise in Large Language Models](https://mlsanity.com/paper/2408.15915/) (2024)

## Abstract (arXiv)

Recent work by Brock et al. (2018) suggests that Generative Adversarial Networks (GANs) benefit disproportionately from large mini-batch sizes. Unfortunately, using large batches is slow and expensive on conventional hardware. Thus, it would be nice if we could generate batches that were effectively large though actually small. In this work, we propose a method to do this, inspired by the use of Coreset-selection in active learning. When training a GAN, we draw a large batch of samples from the prior and then compress that batch using Coreset-selection. To create effectively large batches of 'real' images, we create a cached dataset of Inception activations of each training image, randomly project them down to a smaller dimension, and then use Coreset-selection on those projected activations at training time. We conduct experiments showing that this technique substantially reduces training time and memory usage for modern GAN variants, that it reduces the fraction of dropped modes in a synthetic dataset, and that it allows GANs to reach a new state of the art in anomaly detection.