mlsanityData to 14 Oct 2026

ParaGAN: A Scalable Distributed Training Framework for Generative Adversarial Networks

arXiv 2411.03999 ↗First posted 6 Nov 2024cs.DC cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advances in Generative Artificial Intelligence have fueled numerous applications, particularly those involving Generative Adversarial Networks (GANs), which are essential for synthesizing realistic photos and videos. However, efficiently training GANs remains a critical challenge due to their computationally intensive and numerically unstable nature. Existing methods often require days or even weeks for training, posing significant resource and time constraints. In this work, we introduce ParaGAN, a scalable distributed GAN training framework that leverages asynchronous training and an asymmetric optimization policy to accelerate GAN training. ParaGAN employs a congestion-aware data pipeline and hardware-aware layout transformation to enhance accelerator utilization, resulting in over 30% improvements in throughput. With ParaGAN, we reduce the training time of BigGAN from 15 days to 14 hours while achieving 91% scaling efficiency. Additionally, ParaGAN enables unprecedented high-resolution image generation using BigGAN.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 38 references that are in the map. First posted on arXiv 6 Nov 2024.

Builds on

23 of 38 references.

References · 23

  1. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  3. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  4. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  5. 2018Self-Attention Generative Adversarial Networkscited by 809
  6. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
  7. 2018Which Training Methods for GANs do actually Converge?cited by 679
  8. 2018Generative Image Inpainting with Contextual Attentioncited by 580
  9. 2017Large Batch Training of Convolutional Networkscited by 535
  10. 2019Lookahead Optimizer: k steps forward, 1 step backcited by 336
  11. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  12. 2016Unrolled Generative Adversarial Networkscited by 208
11 more
  1. 2017VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learningcited by 200
  2. 2018The Mechanics of n-Player Differentiable Gamescited by 118
  3. 2017The Numerics of GANscited by 105
  4. 2018The Limit Points of (Optimistic) Gradient Descent in Min-Max Optimizationcited by 101
  5. 2018Patch-Based Image Inpainting with Generative Adversarial Networkscited by 70
  6. 2018MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasetscited by 49
  7. 2019A Decentralized Parallel Algorithm for Training Generative Adversarial Netscited by 43
  8. 2020Synthetic Learning: Learn From Distributed Asynchronized Discriminator GAN Without Sharing Medical Image Datacited by 38
  9. 2018Convergence Problems with Generative Adversarial Networks (GANs)cited by 14
  10. 2020Mimicry: Towards the Reproducibility of GAN Researchcited by 14
  11. 2019Highly-scalable, physics-informed GANs for learning solutions of stochastic PDEscited by 11

Built on by

Not cited yet.