mlsanityData to 7 Oct 2026

Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?

arXiv 2107.00166 ↗First posted 1 Jul 2021cs.LG cs.AI cs.CVcited by 52

Not on the map. Why?

Abstract (arXiv)

There have been long-standing controversies and inconsistencies over the experiment setup and criteria for identifying the "winning ticket" in literature. To reconcile such, we revisit the definition of lottery ticket hypothesis, with comprehensive and more rigorous conditions. Under our new definition, we show concrete evidence to clarify whether the winning ticket exists across the major DNN architectures and/or applications. Through extensive experiments, we perform quantitative analysis on the correlations between winning tickets and various experimental factors, and empirically study the patterns of our observations. We find that the key training hyperparameters, such as learning rate and training epochs, as well as the architecture characteristics such as capacities and residual connections, are all highly correlated with whether and when the winning tickets can be identified. Based on our analysis, we summarize a guideline for parameter settings in regards of specific architecture characteristics, which we hope to catalyze the research progress on the topic of lottery ticket hypothesis. Our codes are publicly available at: https://github.com/boone891214/sanity-check-LTH.

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Builds on 29 of its 38 references that are in the map. First posted on arXiv 1 Jul 2021.

Builds on

29 of 38 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  4. 2016Wide Residual Networkscited by 3,057
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  7. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  8. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 802
  9. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 627
  10. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  11. 2018Rethinking the Value of Network Pruningcited by 595
  12. 2019The State of Sparsity in Deep Neural Networkscited by 471
17 more
  1. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 467
  2. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  3. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 238
  4. 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 206
  5. 2019Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networkscited by 160
  6. 2021A Unified Lottery Ticket Hypothesis for Graph Neural Networkscited by 118
  7. 2020The Early Phase of Neural Network Trainingcited by 114
  8. 2020When BERT Plays the Lottery, All Tickets Are Winningcited by 114
  9. 2019One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizerscited by 102
  10. 2020The Lottery Tickets Hypothesis for Supervised and Self-supervised Pre-training in Computer Vision Modelscited by 91
  11. 2020EarlyBERT: Efficient BERT Training via Early-bird Lottery Ticketscited by 79
  12. 2019Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLPcited by 73
  13. 2021Playing Lottery Tickets with Vision and Languagecited by 45
  14. 2021Efficient Lottery Ticket Finding: Less Data is Morecited by 44
  15. 2020The Lottery Ticket Hypothesis for Object Recognitioncited by 34
  16. 2021GANs Can Play Lottery Tickets Toocited by 32
  17. 2019Sparse Transfer Learning via Winning Lottery Ticketscited by 27

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 13; 2023: 16; 2024: 11; 2025: 5; 2026: 2.

52 papers.

Sort:

By area:

  1. 2023Model Sparsity Can Simplify Machine Unlearningcited by 206
  2. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  3. 2021SPViT: Enabling Faster Vision Transformers via Soft Token Pruningcited by 165
  4. 2024What makes unlearning hard and what to do about itcited by 89
  5. 2021MEST: Accurate and Fast Memory-Economic Sparse Training Framework on the Edgecited by 77
  6. 2023DeepZero: Scaling up Zeroth-Order Optimization for Deep Model Trainingcited by 76
  7. 2022Advancing Model Pruning via Bi-level Optimizationcited by 55
  8. 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 54
  9. 2023Exploring the Landscape of Machine Unlearning: A Comprehensive Survey and Taxonomycited by 44
  10. 2022Fortuitous Forgetting in Connectionist Networkscited by 39
  11. 2023An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learningcited by 39
  12. 2022CHEX: CHannel EXploration for CNN Model Compressioncited by 36
38 more
  1. 2022Rare Gems: Finding Lottery Tickets at Initializationcited by 34
  2. 2022Coarsening the Granularity: Towards Structurally Sparse Lottery Ticketscited by 27
  3. 2022F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantizationcited by 26
  4. 2022Why Random Pruning Is All We Need to Start Sparsecited by 26
  5. 2023Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researcherscited by 25
  6. 2023Rethinking Graph Lottery Tickets: Graph Sparsity Matterscited by 23
  7. 2024A Survey of Lottery Ticket Hypothesiscited by 23
  8. 2022Most Activation Functions Can Win the Lottery Without Excessive Depthcited by 21
  9. 2023Breaking the Trilemma of Privacy, Utility, Efficiency via Controllable Machine Unlearningcited by 21
  10. 2022Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networkscited by 18
  11. 2022Convolutional and Residual Networks Provably Contain Lottery Ticketscited by 16
  12. 2024Masks, Signs, And Learning Rate Rewindingcited by 15
  13. 2023The Snowflake Hypothesis: Training Deep GNN with One Node One Receptive fieldcited by 11
  14. 2023Successfully Applying Lottery Ticket Hypothesis to Diffusion Modelcited by 11
  15. 2024Two Heads Are Better Than One: Boosting Graph Sparse Training via Semantic and Topological Awarenesscited by 10
  16. 2024Revisiting Machine Unlearning with Dimensional Alignmentcited by 10
  17. 2023Unlearning with Fisher Maskingcited by 8
  18. 2022Data Level Lottery Ticket Hypothesis for Vision Transformerscited by 7
  19. 2024DynST: Dynamic Sparse Training for Resource-Constrained Spatio-Temporal Forecastingcited by 7
  20. 2025Sign-In to the Lottery: Reparameterizing Sparse Training From Scratchcited by 6
  21. 2024Accelerating ViT Inference on FPGA through Static and Dynamic Pruningcited by 5
  22. 2021Lottery Tickets with Nonzero Biasescited by 4
  23. 2022Can We Find Strong Lottery Tickets in Generative Models?cited by 4
  24. 2025CoUn: Empowering Machine Unlearning via Contrastive Learningcited by 4
  25. 2022COLT: Cyclic Overlapping Lottery Tickets for Faster Pruning of Convolutional Neural Networkscited by 3
  26. 2023PriPrune: Quantifying and Preserving Privacy in Pruned Federated Learningcited by 3
  27. 2025Not All Explanations for Deep Learning Phenomena Are Equally Valuablecited by 3
  28. 2023Sparse Weight Averaging with Multiple Particles for Iterative Magnitude Pruningcited by 2
  29. 2024Discriminative Adversarial Unlearningcited by 2
  30. 2023Can Adversarial Examples Be Parsed to Reveal Victim Model Information?cited by 1
  31. 2023Distilled Pruning: Using Synthetic Data to Win the Lotterycited by 1
  32. 2024Cyclic Sparse Training: Is it Enough?cited by 1
  33. 2021Toward Compact Parameter Representations for Architecture-Agnostic Neural Network Compressioncited by 0
  34. 2023Unveiling Invariances via Neural Network Pruningcited by 0
  35. 2024Efficient DNN-Powered Software with Fair Sparse Modelscited by 0
  36. 2024Fast Track to Winning Tickets: Repowering One-Shot Pruning for Graph Neural Networkscited by 0
  37. 2025Hyperflux: Pruning Reveals Importancecited by 0
  38. 2025Find A Winning Sign: Sign Is All We Need to Win the Lotterycited by 0

50 most cited of 52 All 52 (JSON)