# Evaluating Lottery Tickets Under Distributional Shifts

arXiv: https://arxiv.org/abs/1910.12708 · HTML: https://mlsanity.com/paper/1910.12708/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2019.

## Built on by (most cited first, 23 of 23)

- [Compacter: Efficient Low-Rank Hypercomplex Adapter Layers](https://mlsanity.com/paper/2106.04647/) (2021)
- [Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization](https://mlsanity.com/paper/2105.12002/) (2021)
- [Efficient Lottery Ticket Finding: Less Data is More](https://mlsanity.com/paper/2106.03225/) (2021)
- [GANs Can Play Lottery Tickets Too](https://mlsanity.com/paper/2106.00134/) (2021)
- [RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratch](https://mlsanity.com/paper/2205.15043/) (2022)
- [XPrompt: Exploring the Extreme of Prompt Tuning](https://mlsanity.com/paper/2210.04457/) (2022)
- [Exploring Lottery Ticket Hypothesis in Spiking Neural Networks](https://mlsanity.com/paper/2207.01382/) (2022)
- [Data-Efficient GAN Training Beyond (Just) Augmentations: A Lottery Ticket Perspective](https://mlsanity.com/paper/2103.00397/) (2021)
- [On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learning](https://mlsanity.com/paper/2105.01648/) (2021)
- [A Survey of Lottery Ticket Hypothesis](https://mlsanity.com/paper/2403.04861/) (2024)
- [The Impact of Differential Privacy on Group Disparity Mitigation](https://mlsanity.com/paper/2203.02745/) (2022)
- [Universality of Winning Tickets: A Renormalization Group Perspective](https://mlsanity.com/paper/2110.03210/) (2021)
- [Exploring the Impact of Model Scaling on Parameter-Efficient Tuning](https://mlsanity.com/paper/2306.02320/) (2023)
- [LilNetX: Lightweight Networks with EXtreme Model Compression and Structured Sparsification](https://mlsanity.com/paper/2204.02965/) (2022)
- [Data Level Lottery Ticket Hypothesis for Vision Transformers](https://mlsanity.com/paper/2211.01484/) (2022)
- [Data-Efficient Double-Win Lottery Tickets from Robust Pre-training](https://mlsanity.com/paper/2206.04762/) (2022)
- [Exploring Out-of-Distribution Generalization in Text Classifiers Trained on Tobacco-3482 and RVL-CDIP](https://mlsanity.com/paper/2108.02684/) (2021)
- [Considering Layerwise Importance in the Lottery Ticket Hypothesis](https://mlsanity.com/paper/2302.11244/) (2023)
- [Value-Based Deep Multi-Agent Reinforcement Learning with Dynamic Sparse Training](https://mlsanity.com/paper/2409.19391/) (2024)
- [On How Iterative Magnitude Pruning Discovers Local Receptive Fields in Fully Connected Neural Networks](https://mlsanity.com/paper/2412.06545/) (2024)
- [Where to Begin: Efficient Pretraining via Subnetwork Selection and Distillation](https://mlsanity.com/paper/2510.07227/) (2025)
- [Neural Network Panning: Screening the Optimal Sparse Network Before Training](https://mlsanity.com/paper/2209.13378/) (2022)
- [Chasing Better Deep Image Priors between Over- and Under-parameterization](https://mlsanity.com/paper/2410.24187/) (2024)

## Abstract (arXiv)

The Lottery Ticket Hypothesis suggests large, over-parameterized neural networks consist of small, sparse subnetworks that can be trained in isolation to reach a similar (or better) test accuracy. However, the initialization and generalizability of the obtained sparse subnetworks have been recently called into question. Our work focuses on evaluating the initialization of sparse subnetworks under distributional shifts. Specifically, we investigate the extent to which a sparse subnetwork obtained in a source domain can be re-trained in isolation in a dissimilar, target domain. In addition, we examine the effects of different initialization strategies at transfer-time. Our experiments show that sparse subnetworks obtained through lottery ticket training do not simply overfit to particular domains, but rather reflect an inductive bias of deep neural networks that can be exploited in multiple domains.