mlsanityData to 7 Oct 2026

Insights into the Lottery Ticket Hypothesis and Iterative Magnitude Pruning

arXiv 2403.15022 ↗First posted 22 Mar 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Lottery ticket hypothesis for deep neural networks emphasizes the importance of initialization used to re-train the sparser networks obtained using the iterative magnitude pruning process. An explanation for why the specific initialization proposed by the lottery ticket hypothesis tends to work better in terms of generalization (and training) performance has been lacking. Moreover, the underlying principles in iterative magnitude pruning, like the pruning of smaller magnitude weights and the role of the iterative process, lack full understanding and explanation. In this work, we attempt to provide insights into these phenomena by empirically studying the volume/geometry and loss landscape characteristics of the solutions obtained at various stages of the iterative magnitude pruning process.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 33 references that are in the map. First posted on arXiv 22 Mar 2024.

Builds on

23 of 33 references.

References · 23

  1. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  2. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  3. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  4. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  5. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  6. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  7. 2019Linear Mode Connectivity and the Lottery Ticket Hypothesiscited by 627
  8. 2018Rethinking the Value of Network Pruningcited by 595
  9. 2020What is the State of Neural Network Pruning?cited by 578
  10. 2017Sharp Minima Can Generalize For Deep Netscited by 462
  11. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  12. 2020Comparing Rewinding and Fine-tuning in Neural Network Pruningcited by 238
11 more
  1. 2020Pruning Neural Networks at Initialization: Why are We Missing the Mark?cited by 181
  2. 2017Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapescited by 84
  3. 2019Understanding Generalization through Visualizationscited by 54
  4. 2018The loss landscape of overparameterized neural networkscited by 38
  5. 2022Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?cited by 38
  6. 2020On the Predictability of Pruning Across Scalescited by 33
  7. 2022Pruning's Effect on Generalization Through the Lens of Training and Regularizationcited by 31
  8. 2021How many degrees of freedom do we need to train deep networks: a loss landscape perspectivecited by 28
  9. 2022Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networkscited by 21
  10. 2019Dissecting Pruned Neural Networkscited by 7
  11. 2021Studying the Consistency and Composability of Lottery Ticket Pruning Maskscited by 1

Built on by

Not cited yet.