mlsanityData to 14 Oct 2026

Convolutional and Residual Networks Provably Contain Lottery Tickets

arXiv 2205.02343 ↗First posted 4 May 2022cs.LG cs.AIcited by 16

Not on the map. Why?

Abstract (arXiv)

The Lottery Ticket Hypothesis continues to have a profound practical impact on the quest for small scale deep neural networks that solve modern deep learning tasks at competitive performance. These lottery tickets are identified by pruning large randomly initialized neural networks with architectures that are as diverse as their applications. Yet, theoretical insights that attest their existence have been mostly focused on deep fully-connected feed forward networks with ReLU activation functions. We prove that also modern architectures consisting of convolutional and residual layers that can be equipped with almost arbitrary activation functions can contain lottery tickets with high probability.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 33 of its 47 references that are in the map. First posted on arXiv 4 May 2022.

Builds on

33 of 47 references.

References · 33

  1. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  2. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  3. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  4. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  5. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 637
  6. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  7. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
  8. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  9. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 240
  10. 2019What's Hidden in a Randomly Weighted Neural Network?cited by 237
  11. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 221
  12. 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 207
21 more
  1. 2020Proving the Lottery Ticket Hypothesis: Pruning is All You Needcited by 201
  2. 2020Pruning Neural Networks at Initialization: Why are We Missing the Mark?cited by 182
  3. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
  4. 2019Drawing Early-Bird Tickets: Towards More Efficient Training of Deep Networkscited by 160
  5. 2021A Unified Lottery Ticket Hypothesis for Graph Neural Networkscited by 118
  6. 2019Winning the Lottery with Continuous Sparsificationcited by 100
  7. 2019A Signal Propagation Perspective for Pruning Neural Networks at Initializationcited by 96
  8. 2021Sparse Training via Boosting Pruning Plasticity with Neuroregenerationcited by 84
  9. 2020Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficientcited by 81
  10. 2020Logarithmic Pruning is All You Needcited by 71
  11. 2020Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpotcited by 66
  12. 2021Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Networkcited by 55
  13. 2021Sanity Checks for Lottery Tickets: Does Your Winning Ticket Really Win the Jackpot?cited by 52
  14. 2021Efficient Lottery Ticket Finding: Less Data is Morecited by 44
  15. 2021On the Existence of Universal Lottery Ticketscited by 30
  16. 2020Pruning via Iterative Ranking of Sensitivity Statisticscited by 26
  17. 2021Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspectivecited by 25
  18. 2021The Elastic Lottery Ticket Hypothesiscited by 23
  19. 2021Lottery Ticket Preserves Weight Correlation: Is It Desirable or Not?cited by 22
  20. 2021Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networkscited by 14
  21. 2021Plant 'n' Seek: Can You Find the Winning Ticket?cited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 1; 2024: 5; 2025: 3; 2026: 4.

16 papers.

Sort:

  1. 2022Why Random Pruning Is All We Need to Start Sparsecited by 26
  2. 2024A Survey of Lottery Ticket Hypothesiscited by 24
  3. 2024Spectral Graph Pruning Against Over-Squashing and Over-Smoothingcited by 24
  4. 2022A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesiscited by 19
  5. 2024Mask in the Mirror: Implicit Sparsificationcited by 14
  6. 2025Sign-In to the Lottery: Reparameterizing Sparse Training From Scratchcited by 6
  7. 2022Strong Lottery Ticket Hypothesis with varepsilon--perturbationcited by 3
  8. 2023Polynomially Over-Parameterized Convolutional Neural Networks Contain Structured Strong Winning Lottery Ticketscited by 3
  9. 2024On the Sparsity of the Strong Lottery Ticket Hypothesiscited by 3
  10. 2025The Graphon Limit Hypothesis: Understanding Neural Network Pruning via Infinite Width Analysiscited by 2
  11. 2024Cyclic Sparse Training: Is it Enough?cited by 1
  12. 2025Quantization vs Pruning: Insights from the Strong Lottery Ticket Hypothesiscited by 1
4 more
  1. 2026Structured vs. Unstructured Pruning: An Exponential Gapcited by 0
  2. 2026HORST: Composing Optimizer Geometries for Sparse Transformer Trainingcited by 0
  3. 2026A Unified Framework for Quantized and Continuous Strong Lottery Ticketscited by 0
  4. 2026Stabilizing the Dynamic Low-Rank Trainingcited by 0