mlsanityData to 14 Oct 2026

On the Existence of Universal Lottery Tickets

arXiv 2111.11146 ↗First posted 22 Nov 2021cs.LG cs.AI stat.MLcited by 30

Not on the map. Why?

Abstract (arXiv)

The lottery ticket hypothesis conjectures the existence of sparse subnetworks of large randomly initialized deep neural networks that can be successfully trained in isolation. Recent work has experimentally observed that some of these tickets can be practically reused across a variety of tasks, hinting at some form of universality. We formalize this concept and theoretically prove that not only do such universal tickets exist but they also do not require further training. Our proofs introduce a couple of technical innovations related to pruning for strong lottery tickets, including extensions of subset sum results and a strategy to leverage higher amounts of depth. Our explicit sparse constructions of universal function families might be of independent interest, as they highlight representational benefits induced by univariate convolutional architectures.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 25 of its 37 references that are in the map. First posted on arXiv 22 Nov 2021.

Builds on

25 of 37 references.

References · 25

  1. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  2. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  3. 2018Reconciling modern machine learning practice and the bias-variance trade-offcited by 927
  4. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  5. 2014Deep Learning in Neural Networks: An Overviewcited by 770
  6. 2016Error bounds for approximations with deep ReLU networkscited by 574
  7. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
  8. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  9. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 433
  10. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  11. 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
  12. 2016Benefits of depth in neural networkscited by 250
13 more
  1. 2019What's Hidden in a Randomly Weighted Neural Network?cited by 237
  2. 2020Proving the Lottery Ticket Hypothesis: Pruning is All You Needcited by 201
  3. 2018Optimal approximation of continuous functions by very deep ReLU networkscited by 166
  4. 2017The Shattered Gradients Problem: If resnets are the answer, then what is the question?cited by 157
  5. 2020Dynamic Model Pruning with Feedbackcited by 125
  6. 2019One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizerscited by 102
  7. 2019Winning the Lottery with Continuous Sparsificationcited by 100
  8. 2020The Cost of Training NLP Models: A Concise Overviewcited by 98
  9. 2020The Kolmogorov-Arnold representation theorem revisitedcited by 89
  10. 2020Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficientcited by 81
  11. 2020Logarithmic Pruning is All You Needcited by 71
  12. 2021Multi-Prize Lottery Ticket Hypothesis: Finding Accurate Binary Neural Networks by Pruning A Randomly Weighted Networkcited by 55
  13. 2019Nonlinear Approximation and (Deep) ReLU Networkscited by 45

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 8; 2024: 8; 2025: 2; 2026: 4.

30 papers.

Sort:

By area:

  1. 2022Exploring Lottery Ticket Hypothesis in Spiking Neural Networkscited by 26
  2. 2022Why Random Pruning Is All We Need to Start Sparsecited by 26
  3. 2024A Survey of Lottery Ticket Hypothesiscited by 24
  4. 2024Spectral Graph Pruning Against Over-Squashing and Over-Smoothingcited by 24
  5. 2022Most Activation Functions Can Win the Lottery Without Excessive Depthcited by 21
  6. 2023Sampling weights of deep neural networkscited by 20
  7. 2022A General Framework For Proving The Equivariant Strong Lottery Ticket Hypothesiscited by 19
  8. 2022Convolutional and Residual Networks Provably Contain Lottery Ticketscited by 16
  9. 2024Masks, Signs, And Learning Rate Rewindingcited by 15
  10. 2024Mask in the Mirror: Implicit Sparsificationcited by 14
  11. 2022Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Surveycited by 12
  12. 2023Successfully Applying Lottery Ticket Hypothesis to Diffusion Modelcited by 11
18 more
  1. 2023Efficient Neural Networks for Tiny Machine Learning: A Comprehensive Reviewcited by 9
  2. 2024Optimal Recurrent Network Topologies for Dynamical Systems Reconstructioncited by 8
  3. 2025Sign-In to the Lottery: Reparameterizing Sparse Training From Scratchcited by 6
  4. 2023Implicit Compressibility of Overparametrized Neural Networks Trained with Heavy-Tailed SGDcited by 5
  5. 2026Fixed Aggregation Features Can Rival GNNscited by 4
  6. 2022Strong Lottery Ticket Hypothesis with varepsilon--perturbationcited by 3
  7. 2024On the Sparsity of the Strong Lottery Ticket Hypothesiscited by 3
  8. 2022DASS: Differentiable Architecture Search for Sparse neural networkscited by 2
  9. 2023Polarity is all you need to learn and transfer fastercited by 2
  10. 2023Pruning at Initialization -- A Sketching Perspectivecited by 2
  11. 2024Cyclic Sparse Training: Is it Enough?cited by 1
  12. 2026Routing the Lottery: Adaptive Subnetworks for Heterogeneous Datacited by 1
  13. 2023Learning to Learn with Indispensable Connectionscited by 0
  14. 2023Transferability of Winning Lottery Tickets in Neural Network Differential Equation Solverscited by 0
  15. 2024A simple theory for training response of deep neural networkscited by 0
  16. 2025The Quest for Winning Tickets in Low-Rank Adapterscited by 0
  17. 2026Structured vs. Unstructured Pruning: An Exponential Gapcited by 0
  18. 2026Man, Machine, and Mathematicscited by 0