mlsanityData to 7 Oct 2026

Can We Find Strong Lottery Tickets in Generative Models?

arXiv 2212.08311 ↗First posted 16 Dec 2022cs.CV cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Yes. In this paper, we investigate strong lottery tickets in generative models, the subnetworks that achieve good generative performance without any weight update. Neural network pruning is considered the main cornerstone of model compression for reducing the costs of computation and memory. Unfortunately, pruning a generative model has not been extensively explored, and all existing pruning algorithms suffer from excessive weight-training costs, performance degradation, limited generalizability, or complicated training. To address these problems, we propose to find a strong lottery ticket via moment-matching scores. Our experimental results show that the discovered subnetwork can perform similarly or better than the trained dense model even when only 10% of the weights remain. To the best of our knowledge, we are the first to show the existence of strong lottery tickets in generative models and provide an algorithm to find it stably. Our code and supplementary materials are publicly available.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 46 of its 56 references that are in the map. First posted on arXiv 16 Dec 2022.

Builds on

46 of 56 references.

References · 46

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  10. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  11. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  12. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
34 more
  1. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  2. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  3. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  4. 2018Demystifying MMD GANscited by 1,268
  5. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  6. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  7. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  8. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 627
  9. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
  10. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
  11. 2020Reliable Fidelity and Diversity Metrics for Generative Modelscited by 425
  12. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  13. 2015Generative Moment Matching Networkscited by 252
  14. 2017MMD GAN: Towards Deeper Understanding of Moment Matching Networkcited by 251
  15. 2019What's Hidden in a Randomly Weighted Neural Network?cited by 236
  16. 2019Few-Shot Unsupervised Image-to-Image Translationcited by 222
  17. 2020Pruning Neural Networks at Initialization: Why are We Missing the Mark?cited by 181
  18. 2021TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Upcited by 162
  19. 2020Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpotcited by 66
  20. 2020Rethinking the Truly Unsupervised Image-to-Image Translationcited by 56
  21. 2021Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?cited by 52
  22. 2022Revisiting Random Channel Pruning for Neural Network Compressioncited by 50
  23. 2020GAN Slimming: All-in-One GAN Compression by A Unified Optimization Frameworkcited by 44
  24. 2022StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesiscited by 38
  25. 2022Rare Gems: Finding Lottery Tickets at Initializationcited by 34
  26. 2021GANs Can Play Lottery Tickets Toocited by 32
  27. 2021Content-Aware GAN Compressioncited by 28
  28. 2020Slimmable Generative Adversarial Networkscited by 27
  29. 2019DeepCABAC: A Universal Compression Algorithm for Deep Neural Networkscited by 26
  30. 2020Winning Lottery Tickets in Deep Generative Modelscited by 25
  31. 2021Revisiting Discriminator in GAN Compression: A Generator-discriminator Cooperative Compression Schemecited by 23
  32. 2018Improving MMD-GAN Training with Repulsive Loss Functioncited by 22
  33. 2019Learning Implicit Generative Models by Matching Perceptual Featurescited by 18
  34. 2021Generative Optimization Networks for Memory Efficient Data Generationcited by 5

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 1.

4 papers.

  1. 2024A Survey of Lottery Ticket Hypothesiscited by 23
  2. 2025PRISM: Privacy-Preserving Improved Stochastic Masking for Federated Generative Modelscited by 3
  3. 2024Nickel and Diming Your GAN: A Dual-Method Approach to Enhancing GAN Efficiency via Knowledge Distillationcited by 2
  4. 2024Generative adversarial learning with optimal input dimension and its adaptive generator architecturecited by 1