mlsanityData to 7 Oct 2026

Balanced Training for Sparse GANs

arXiv 2302.14670 ↗First posted 28 Feb 2023cs.LG cs.CV stat.MLcited by 8

Not on the map. Why?

Abstract (arXiv)

Over the past few years, there has been growing interest in developing larger and deeper neural networks, including deep generative models like generative adversarial networks (GANs). However, GANs typically come with high computational complexity, leading researchers to explore methods for reducing the training and inference costs. One such approach gaining popularity in supervised learning is dynamic sparse training (DST), which maintains good performance while enjoying excellent training efficiency. Despite its potential benefits, applying DST to GANs presents challenges due to the adversarial nature of the training process. In this paper, we propose a novel metric called the balance ratio (BR) to study the balance between the sparse generator and discriminator. We also introduce a new method called balanced dynamic sparse training (ADAPT), which seeks to control the BR during GAN training to achieve a good trade-off between performance and computational cost. Our proposed method shows promising results on multiple datasets, demonstrating its effectiveness.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 80 of its 95 references that are in the map. First posted on arXiv 28 Feb 2023.

Builds on

80 of 95 references.

References · 80

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2022Generative Adversarial Networkscited by 5,603
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  6. 2016Improved Techniques for Training GANscited by 2,938
  7. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  8. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  9. 2017Improved Training of Wasserstein GANscited by 2,223
  10. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  11. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  12. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
38 more
  1. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  2. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  3. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
  4. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  5. 2017StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translationcited by 1,036
  6. 2016Least Squares Generative Adversarial Networkscited by 1,000
  7. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
  8. 2018Self-Attention Generative Adversarial Networkscited by 809
  9. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  10. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  11. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  12. 2018Rethinking the Value of Network Pruningcited by 595
  13. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 571
  14. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  15. 2019Importance Estimation for Neural Network Pruningcited by 531
  16. 2019Rigging the Lottery: Making All Tickets Winnerscited by 491
  17. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  18. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
  19. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
  20. 2017ThiNet: A Filter Level Pruning Method for Deep Neural Network Compressioncited by 438
  21. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  22. 2020Differentiable Augmentation for Data-Efficient GAN Trainingcited by 361
  23. 2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 325
  24. 2017Toward Multimodal Image-to-Image Translationcited by 295
  25. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 238
  26. 2016Dynamic Network Surgery for Efficient DNNscited by 233
  27. 2017NISP: Pruning Networks using Neuron Importance Score Propagationcited by 228
  28. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 219
  29. 2019Sparse Networks from Scratch: Faster Training without Losing Performancecited by 210
  30. 2019Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterizationcited by 187
  31. 2017Data-Driven Sparse Structure Selection for Deep Neural Networkscited by 181
  32. 2018Low-Shot Learning from Imaginary Datacited by 181
  33. 2020Pruning Neural Networks at Initialization: Why are We Missing the Mark?cited by 181
  34. 2017Generalization and Equilibrium in Generative Adversarial Nets (GANs)cited by 165
  35. 2020Soft Threshold Weight Reparameterization for Learnable Sparsitycited by 150
  36. 2017Deep Rewiring: Training very sparse deep networkscited by 136
  37. 2017BEGAN: Boundary Equilibrium Generative Adversarial Networkscited by 121
  38. 2021Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Trainingcited by 107

50 most cited of 80 All 80 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 7; 2025: 1.

8 papers.

  1. 2024Residual-based Language Models are Free Boosters for Biomedical Imagingcited by 23
  2. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  3. 2024The New Agronomists: Language Models are Experts in Crop Managementcited by 12
  4. 2024Mapping New Realities: Ground Truth Image Creation with Pix2Pix Image-to-Image Translationcited by 7
  5. 2024BadCLM: Backdoor Attack in Clinical Language Models for Electronic Health Recordscited by 5
  6. 2024A Survey on the Application of Generative Adversarial Networks in Cybersecurity: Prospective, Direction and Open Research Scopescited by 2
  7. 2024Deep Representation Learning for Multi-functional Degradation Modeling of Community-dwelling Aging Populationcited by 1
  8. 2025Bridging GANs and Bayesian Neural Networks via Partial Stochasticitycited by 0