# Studying the Consistency and Composability of Lottery Ticket Pruning Masks

arXiv: https://arxiv.org/abs/2104.14753 · HTML: https://mlsanity.com/paper/2104.14753/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 5 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 5 of 5)

- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Linear Mode Connectivity and the Lottery Ticket Hypothesis](https://mlsanity.com/paper/1912.05671/) (2019)
- [Pruning Neural Networks at Initialization: Why are We Missing the Mark?](https://mlsanity.com/paper/2009.08576/) (2020)
- [Bespoke vs. Pret-`a-Porter Lottery Tickets: Exploiting Mask Similarity for Trainable Sub-Network Finding](https://mlsanity.com/paper/2007.04091/) (2020)

## Built on by (most cited first, 1 of 1)

- [Insights into the Lottery Ticket Hypothesis and Iterative Magnitude Pruning](https://mlsanity.com/paper/2403.15022/) (2024)

## Abstract (arXiv)

Magnitude pruning is a common, effective technique to identify sparse subnetworks at little cost to accuracy. In this work, we ask whether a particular architecture's accuracy-sparsity tradeoff can be improved by combining pruning information across multiple runs of training. From a shared ResNet-20 initialization, we train several network copies (siblings) to completion using different SGD data orders on CIFAR-10. While the siblings' pruning masks are naively not much more similar than chance, starting sibling training after a few epochs of shared pretraining significantly increases pruning overlap. We then choose a subnetwork by either (1) taking all weights that survive pruning in any sibling (mask union), or (2) taking only the weights that survive pruning across all siblings (mask intersection). The resulting subnetwork is retrained. Strikingly, we find that union and intersection masks perform very similarly. Both methods match the accuracy-sparsity tradeoffs of the one-shot magnitude pruning baseline, even when we combine masks from up to k = 10 siblings.