mlsanityData to 7 Oct 2026

A Survey of Lottery Ticket Hypothesis

arXiv 2403.04861 ↗First posted 7 Mar 2024cs.LG cs.NEcited by 23

Not on the map. Why?

Abstract (arXiv)

The Lottery Ticket Hypothesis (LTH) states that a dense neural network model contains a highly sparse subnetwork (i.e., winning tickets) that can achieve even better performance than the original model when trained in isolation. While LTH has been proved both empirically and theoretically in many works, there still are some open issues, such as efficiency and scalability, to be addressed. Also, the lack of open-source frameworks and consensual experimental setting poses a challenge to future research on LTH. We, for the first time, examine previous research and studies on LTH from different perspectives. We also discuss issues in existing works and list potential directions for further exploration. This survey aims to provide an in-depth look at the state of LTH and develop a duly maintained platform to conduct experiments and compare with the most updated baselines.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 100 of its 123 references that are in the map. First posted on arXiv 7 Mar 2024.

Builds on

100 of 123 references.

References · 100

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  4. 2017Inductive Representation Learning on Large Graphscited by 6,284
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  7. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  8. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  9. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  10. 2018Rethinking the Value of Network Pruningcited by 595
  11. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  12. 2019Rigging the Lottery: Making All Tickets Winnerscited by 491
38 more
  1. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 475
  2. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
  3. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  4. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  5. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 238
  6. 2019What's Hidden in a Randomly Weighted Neural Network?cited by 236
  7. 2016Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architecturescited by 220
  8. 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 206
  9. 2020Proving the Lottery Ticket Hypothesis: Pruning is All You Needcited by 200
  10. 2019Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networkscited by 160
  11. 2018A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multiplierscited by 118
  12. 2021A Unified Lottery Ticket Hypothesis for Graph Neural Networkscited by 118
  13. 2020When BERT Plays the Lottery, All Tickets Are Winningcited by 114
  14. 2019One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizerscited by 102
  15. 2019Winning the Lottery with Continuous Sparsificationcited by 99
  16. 2020The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Modelscited by 91
  17. 2020Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficientcited by 81
  18. 2020EarlyBERT: Efficient BERT Training via Early-bird Lottery Ticketscited by 79
  19. 2020Gradient Flow in Sparse Neural Networks and How Lottery Tickets Wincited by 77
  20. 2021MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edgecited by 77
  21. 2019Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLPcited by 73
  22. 2020Logarithmic Pruning is All You Needcited by 71
  23. 2020Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpotcited by 66
  24. 2020LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasetscited by 60
  25. 2021Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalizationcited by 56
  26. 2021Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Networkcited by 55
  27. 2021Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?cited by 52
  28. 2021How Well Do Sparse Imagenet Models Transfer?cited by 50
  29. 2020Sparse Weight Activation Trainingcited by 45
  30. 2021Playing Lottery Tickets with Vision and Languagecited by 45
  31. 2021Efficient Lottery Ticket Finding: Less Data is Morecited by 44
  32. 2018ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multiplierscited by 42
  33. 2022Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equationscited by 42
  34. 2021When to Prune? A Policy towards Early Structural Pruningcited by 38
  35. 2022Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?cited by 38
  36. 2019Stabilizing the Lottery Ticket Hypothesiscited by 36
  37. 2022CHEX: CHannel EXploration for CNN Model Compressioncited by 36
  38. 2020The Lottery Ticket Hypothesis for Object Recognitioncited by 34

50 most cited of 100 All 100 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 11; 2026: 8.

23 papers.

Sort:

By area:

  1. 2025Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?cited by 32
  2. 2024Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generationcited by 20
  3. 2024Mask in the Mirror: Implicit Sparsificationcited by 12
  4. 2025SAFE: Finding Sparse and Flat Minima to Improve Pruningcited by 5
  5. 2026When Shared Knowledge Hurts: Spectral Over-Accumulation in Model Mergingcited by 5
  6. 2026Your Language Model Secretly Contains Personality Subnetworkscited by 4
  7. 2024On the Sparsity of the Strong Lottery Ticket Hypothesiscited by 3
  8. 2025Flow to Learn: Flow Matching on Neural Network Parameterscited by 3
  9. 2025Neuroplasticity in Artificial Intelligence -- An Overview and Inspirations on Drop In & Out Learningcited by 3
  10. 2025Not All Explanations for Deep Learning Phenomena Are Equally Valuablecited by 3
  11. 2025Expand Neurons, Not Parameterscited by 2
  12. 2025SliceFine: The Universal Winning-Slice Hypothesis for Pretrained Networkscited by 2
11 more
  1. 2026Bayesian Lottery Ticket Hypothesiscited by 2
  2. 2024A Comparative Study of Pruning Methods in Transformer-based Time Series Forecastingcited by 1
  3. 2025Flows and Diffusions on the Neural Manifoldcited by 1
  4. 2026It's Not a Lottery, It's a Race: Understanding How Gradient Descent Adapts the Network's Capacity to the Taskcited by 1
  5. 2026Invariant Learning Dynamics of Transformers in Inductive Reasoning Taskscited by 1
  6. 2025Weisfeiler and Leman Go Gambling: Why Expressive Lottery Tickets Wincited by 0
  7. 2025Investigating the Lottery Ticket Hypothesis for Variational Quantum Circuitscited by 0
  8. 2025A Recovery Guarantee for Sparse Neural Networkscited by 0
  9. 2026Efficient Search of Implantable Adaptive Cells for Medical Image Segmentationcited by 0
  10. 2026Winning Lottery Tickets in Neural Networks via a Quantum-Inspired Classical Algorithmcited by 0
  11. 2026Stabilizing the Dynamic Low-Rank Trainingcited by 0