# On the Transferability of Winning Tickets in Non-Natural Image Datasets

arXiv: https://arxiv.org/abs/2005.05232 · HTML: https://mlsanity.com/paper/2005.05232/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 May 2020.

## Built on by (most cited first, 10 of 10)

- [DiSparse: Disentangled Sparsification for Multitask Model Compression](https://mlsanity.com/paper/2206.04662/) (2022)
- [Universality of Winning Tickets: A Renormalization Group Perspective](https://mlsanity.com/paper/2110.03210/) (2021)
- [Sifting out the features by pruning: Are convolutional networks the winning lottery ticket of fully connected ones?](https://mlsanity.com/paper/2104.13343/) (2021)
- [Strong Lottery Ticket Hypothesis with varepsilon--perturbation](https://mlsanity.com/paper/2210.16589/) (2022)
- [Towards Compute-Optimal Transfer Learning](https://mlsanity.com/paper/2304.13164/) (2023)
- [Pruning at Initialization -- A Sketching Perspective](https://mlsanity.com/paper/2305.17559/) (2023)
- [Sparsity-Driven Plasticity in Multi-Task Reinforcement Learning](https://mlsanity.com/paper/2508.06871/) (2025)
- [Considering Layerwise Importance in the Lottery Ticket Hypothesis](https://mlsanity.com/paper/2302.11244/) (2023)
- [Iterative Magnitude Pruning as a Renormalisation Group: A Study in The Context of The Lottery Ticket Hypothesis](https://mlsanity.com/paper/2308.03128/) (2023)
- [On How Iterative Magnitude Pruning Discovers Local Receptive Fields in Fully Connected Neural Networks](https://mlsanity.com/paper/2412.06545/) (2024)

## Abstract (arXiv)

We study the generalization properties of pruned neural networks that are the winners of the lottery ticket hypothesis on datasets of natural images. We analyse their potential under conditions in which training data is scarce and comes from a non-natural domain. Specifically, we investigate whether pruned models that are found on the popular CIFAR-10/100 and Fashion-MNIST datasets, generalize to seven different datasets that come from the fields of digital pathology and digital heritage. Our results show that there are significant benefits in transferring and training sparse architectures over larger parametrized models, since in all of our experiments pruned networks, winners of the lottery ticket hypothesis, significantly outperform their larger unpruned counterparts. These results suggest that winning initializations do contain inductive biases that are generic to some extent, although, as reported by our experiments on the biomedical datasets, their generalization properties can be more limiting than what has been so far observed in the literature.