# Cats, not CAT scans: a study of dataset similarity in transfer learning for 2D medical image classification

arXiv: https://arxiv.org/abs/2107.05940 · HTML: https://mlsanity.com/paper/2107.05940/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 54 references that are in the map.
- First posted on arXiv 13 Jul 2021.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Describing Textures in the Wild](https://mlsanity.com/paper/1311.3618/) (2013)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1902.03368/) (2019)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [Do Better ImageNet Models Transfer Better?](https://mlsanity.com/paper/1805.08974/) (2018)
- [Rotation Equivariant CNNs for Digital Pathology](https://mlsanity.com/paper/1806.03962/) (2018)
- [Meta-Learning: A Survey](https://mlsanity.com/paper/1810.03548/) (2018)
- [A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises](https://mlsanity.com/paper/2008.09104/) (2020)
- [Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?](https://mlsanity.com/paper/1706.00712/) (2017)
- [Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning](https://mlsanity.com/paper/1602.03409/) (2016)
- [Task2Vec: Task Embedding for Meta-Learning](https://mlsanity.com/paper/1902.03545/) (2019)
- [What makes ImageNet good for transfer learning?](https://mlsanity.com/paper/1608.08614/) (2016)
- [Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis](https://mlsanity.com/paper/1804.06353/) (2018)
- [Carbontracker: Tracking and Predicting the Carbon Footprint of Training Deep Learning Models](https://mlsanity.com/paper/2007.03051/) (2020)
- [Geometric Dataset Distances via Optimal Transport](https://mlsanity.com/paper/2002.02923/) (2020)
- [The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research](https://mlsanity.com/paper/2010.15581/) (2020)
- [Learning to select data for transfer learning with Bayesian Optimization](https://mlsanity.com/paper/1707.05246/) (2017)
- [A scoping review of transfer learning research on medical image analysis using ImageNet](https://mlsanity.com/paper/2004.13175/) (2020)
- [Surrogate Supervision for Medical Image Analysis: Effective Deep Learning From Limited Quantities of Labeled Data](https://mlsanity.com/paper/1901.08707/) (2019)
- [Knowledge Transfer for Melanoma Screening with Deep Learning](https://mlsanity.com/paper/1703.07479/) (2017)
- [Factors of Transferability for a Generic ConvNet Representation](https://mlsanity.com/paper/1406.5774/) (2014)
- [Cats or CAT scans: transfer learning from natural or medical image source datasets?](https://mlsanity.com/paper/1810.05444/) (2018)
- [Learning with less data via Weakly Labeled Patch Classification in Digital Pathology](https://mlsanity.com/paper/1911.12425/) (2019)
- [A Comparative Study of CNN, BoVW and LBP for Classification of Histopathological Images](https://mlsanity.com/paper/1710.01249/) (2017)
- [Multi-source Transfer Learning with Convolutional Neural Networks for Lung Pattern Analysis](https://mlsanity.com/paper/1612.02589/) (2016)
- [Building medical image classifiers with very limited data using segmentation networks](https://mlsanity.com/paper/1808.05205/) (2018)

## Built on by (most cited first, 1 of 1)

- [Revisiting Hidden Representations in Transfer Learning for Medical Imaging](https://mlsanity.com/paper/2302.08272/) (2023)

## Abstract (arXiv)

Transfer learning is a commonly used strategy for medical image classification, especially via pretraining on source data and fine-tuning on target data. There is currently no consensus on how to choose appropriate source data, and in the literature we can find both evidence of favoring large natural image datasets such as ImageNet, and evidence of favoring more specialized medical datasets. In this paper we perform a systematic study with nine source datasets with natural or medical images, and three target medical datasets, all with 2D images. We find that ImageNet is the source leading to the highest performances, but also that larger datasets are not necessarily better. We also study different definitions of data similarity. We show that common intuitions about similarity may be inaccurate, and therefore not sufficient to predict an appropriate source a priori. Finally, we discuss several steps needed for further research in this field, especially with regard to other types (for example 3D) medical images. Our experiments and pretrained models are available via urlhttps://www.github.com/vcheplygina/cats-scans