mlsanityData to 14 Oct 2026

Visualizing the Loss Landscape of Winning Lottery Tickets

arXiv 2112.08538 ↗First posted 16 Dec 2021cs.LG cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

The underlying loss landscapes of deep neural networks have a great impact on their training, but they have mainly been studied theoretically due to computational constraints. This work vastly reduces the time required to compute such loss landscapes, and uses them to study winning lottery tickets found via iterative magnitude pruning. We also share results that contradict previously claimed correlations between certain loss landscape projection methods and model trainability and generalization error.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 13 of its 15 references that are in the map. First posted on arXiv 16 Dec 2021.

Builds on

13 of 15 references.

References · 13

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  4. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  5. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  6. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  7. 2017Exploring Generalization in Deep Learningcited by 600
  8. 2018Rethinking the Value of Network Pruningcited by 599
  9. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  10. 2019Deconstructing Lottery Tickets: Zeros, Signs, and the Supermaskcited by 207
  11. 2019Playing the lottery with rewards and multiple languages: lottery tickets in RL and NLPcited by 73
  12. 2021Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networkscited by 14
1 more
  1. 2021LossPlot: A Better Way to Visualize Loss Landscapescited by 5

Built on by

Cited by year

Citing papers per year: 2022: 2.

2 papers.

  1. 2022Analyzing Lottery Ticket Hypothesis from PAC-Bayesian Theory Perspectivecited by 9
  2. 2022The Role Of Biology In Deep Learningcited by 0