# Cats or CAT scans: transfer learning from natural or medical image source datasets?

arXiv: https://arxiv.org/abs/1810.05444 · HTML: https://mlsanity.com/paper/1810.05444/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2018.

## Built on by (most cited first, 14 of 14)

- [Interpretable and intervenable ultrasonography-based machine learning models for pediatric appendicitis](https://mlsanity.com/paper/2302.14460/) (2023)
- [Probabilistic Integration of Object Level Annotations in Chest X-ray Classification](https://mlsanity.com/paper/2210.06980/) (2022)
- [Source Matters: Source Dataset Impact on Model Robustness in Medical Imaging](https://mlsanity.com/paper/2403.04484/) (2024)
- [TATL: Task Agnostic Transfer Learning for Skin Attributes Detection](https://mlsanity.com/paper/2104.01641/) (2021)
- [Task Fingerprinting for Meta Learning in Biomedical Image Analysis](https://mlsanity.com/paper/2107.03949/) (2021)
- [Addressing materials' microstructure diversity using transfer learning](https://mlsanity.com/paper/2107.13841/) (2021)
- [DRG-Net: Interactive Joint Learning of Multi-lesion Segmentation and Classification for Diabetic Retinopathy Grading](https://mlsanity.com/paper/2212.14615/) (2022)
- [Cats, not CAT scans: a study of dataset similarity in transfer learning for 2D medical image classification](https://mlsanity.com/paper/2107.05940/) (2021)
- [Domain Adaptation using Silver Standard Masks for Lateral Ventricle Segmentation in FLAIR MRI](https://mlsanity.com/paper/2307.08456/) (2023)
- [Exploring connections of spectral analysis and transfer learning in medical imaging](https://mlsanity.com/paper/2407.11379/) (2024)
- [Requirement analysis for an artificial intelligence model for the diagnosis of the COVID-19 from chest X-ray data](https://mlsanity.com/paper/2110.12464/) (2021)
- [Transfer learning from a sparsely annotated dataset of 3D medical images](https://mlsanity.com/paper/2311.05032/) (2023)
- [Lite-FBCN: Lightweight Fast Bilinear Convolutional Network for Brain Disease Classification from MRI Image](https://mlsanity.com/paper/2409.10952/) (2024)
- [Intuitions of Machine Learning Researchers about Transfer Learning for Medical Image Classification](https://mlsanity.com/paper/2510.00902/) (2025)

## Abstract (arXiv)

Transfer learning is a widely used strategy in medical image analysis. Instead of only training a network with a limited amount of data from the target task of interest, we can first train the network with other, potentially larger source datasets, creating a more robust model. The source datasets do not have to be related to the target task. For a classification task in lung CT images, we could use both head CT images, or images of cats, as the source. While head CT images appear more similar to lung CT images, the number and diversity of cat images might lead to a better model overall. In this survey we review a number of papers that have performed similar comparisons. Although the answer to which strategy is best seems to be "it depends", we discuss a number of research directions we need to take as a community, to gain more understanding of this topic.