# Iterative Magnitude Pruning as a Renormalisation Group: A Study in The Context of The Lottery Ticket Hypothesis

arXiv: https://arxiv.org/abs/2308.03128 · HTML: https://mlsanity.com/paper/2308.03128/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 6 Aug 2023.

## Builds on (most cited first, 24 of 24)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask](https://mlsanity.com/paper/1905.01067/) (2019)
- [One ticket to win them all: generalizing lottery ticket initializations across datasets and optimizers](https://mlsanity.com/paper/1906.02773/) (2019)
- [Why does deep and cheap learning work so well?](https://mlsanity.com/paper/1608.08225/) (2016)
- [Are All Layers Created Equal?](https://mlsanity.com/paper/1902.01996/) (2019)
- [An exact mapping between the Variational Renormalization Group and Deep Learning](https://mlsanity.com/paper/1410.3831/) (2014)
- [Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?](https://mlsanity.com/paper/2210.03044/) (2022)
- [Stabilizing the Lottery Ticket Hypothesis](https://mlsanity.com/paper/1903.01611/) (2019)
- [Hamiltonian neural networks for solving equations of motion](https://mlsanity.com/paper/2001.11107/) (2020)
- [Optimizing Neural Networks via Koopman Operator Theory](https://mlsanity.com/paper/2006.02361/) (2020)
- [On the Predictability of Pruning Across Scales](https://mlsanity.com/paper/2006.10621/) (2020)
- [Lottery Tickets in Linear Models: An Analysis of Iterative Magnitude Pruning](https://mlsanity.com/paper/2007.08243/) (2020)
- [The Elastic Lottery Ticket Hypothesis](https://mlsanity.com/paper/2103.16547/) (2021)
- [Towards Understanding Iterative Magnitude Pruning: Why Lottery Tickets Win](https://mlsanity.com/paper/2106.06955/) (2021)
- [On the Transferability of Winning Tickets in Non-Natural Image Datasets](https://mlsanity.com/paper/2005.05232/) (2020)
- [On the Neural Tangent Kernel Analysis of Randomly Pruned Neural Networks](https://mlsanity.com/paper/2203.14328/) (2022)
- [Universality of Winning Tickets: A Renormalization Group Perspective](https://mlsanity.com/paper/2110.03210/) (2021)
- [A Closer Look at Structured Pruning for Neural Network Compression](https://mlsanity.com/paper/1810.04622/) (2018)
- [Dynamical Systems and Neural Networks](https://mlsanity.com/paper/2004.11826/) (2020)
- [Robust Tickets Can Transfer Better: Drawing More Transferable Subnetworks in Transfer Learning](https://mlsanity.com/paper/2304.11834/) (2023)
- [Zeroth-Order Topological Insights into Iterative Magnitude Pruning](https://mlsanity.com/paper/2206.06563/) (2022)
- [Local error quantification for Neural Network Differential Equation solvers](https://mlsanity.com/paper/2008.12190/) (2020)

## Built on by (most cited first, 1 of 1)

- [Man, Machine, and Mathematics](https://mlsanity.com/paper/2604.27052/) (2026)

## Abstract (arXiv)

This thesis delves into the intricate world of Deep Neural Networks (DNNs), focusing on the exciting concept of the Lottery Ticket Hypothesis (LTH). The LTH posits that within extensive DNNs, smaller, trainable subnetworks termed "winning tickets", can achieve performance comparable to the full model. A key process in LTH, Iterative Magnitude Pruning (IMP), incrementally eliminates minimal weights, emulating stepwise learning in DNNs. Once we identify these winning tickets, we further investigate their "universality". In other words, we check if a winning ticket that works well for one specific problem could also work well for other, similar problems. We also bridge the divide between the IMP and the Renormalisation Group (RG) theory in physics, promoting a more rigorous understanding of IMP.