# Evaluating the Robustness of Self-Supervised Learning in Medical Imaging

arXiv: https://arxiv.org/abs/2105.06986 · HTML: https://mlsanity.com/paper/2105.06986/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 14 May 2021.

## Builds on (most cited first, 16 of 16)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)
- [Transfusion: Understanding Transfer Learning for Medical Imaging](https://mlsanity.com/paper/1902.07208/) (2019)
- [Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge](https://mlsanity.com/paper/1612.08012/) (2016)
- [The KiTS19 Challenge Data: 300 Kidney Tumor Cases with Clinical Context, CT Semantic Segmentations, and Surgical Outcomes](https://mlsanity.com/paper/1904.00445/) (2019)
- [Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis](https://mlsanity.com/paper/1908.06912/) (2019)
- [Webly Supervised Learning for Skin Lesion Classification](https://mlsanity.com/paper/1804.00177/) (2018)

## Built on by (most cited first, 13 of 13)

- [Self-supervised Learning from 100 Million Medical Images](https://mlsanity.com/paper/2201.01283/) (2022)
- [MedMNIST-C: Comprehensive benchmark and improved classifier robustness by simulating realistic image corruptions](https://mlsanity.com/paper/2406.17536/) (2024)
- [A Review of Predictive and Contrastive Self-supervised Learning for Medical Images](https://mlsanity.com/paper/2302.05043/) (2023)
- [Focused Decoding Enables 3D Anatomical Detection by Transformers](https://mlsanity.com/paper/2207.10774/) (2022)
- [On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy](https://mlsanity.com/paper/2106.13497/) (2021)
- [Generating and Weighting Semantically Consistent Sample Pairs for Ultrasound Contrastive Learning](https://mlsanity.com/paper/2212.04097/) (2022)
- [Understanding Calibration of Deep Neural Networks for Medical Image Classification](https://mlsanity.com/paper/2309.13132/) (2023)
- [Improving Robustness and Reliability in Medical Image Classification with Latent-Guided Diffusion and Nested-Ensembles](https://mlsanity.com/paper/2310.15952/) (2023)
- [Improved skin lesion recognition by a Self-Supervised Curricular Deep Learning approach](https://mlsanity.com/paper/2112.12086/) (2021)
- [Time-based Self-supervised Learning for Wireless Capsule Endoscopy](https://mlsanity.com/paper/2204.09773/) (2022)
- [Frequency Dropout: Feature-Level Regularization via Randomized Filtering](https://mlsanity.com/paper/2209.09844/) (2022)
- [Dive into Self-Supervised Learning for Medical Image Analysis: Data, Models and Tasks](https://mlsanity.com/paper/2209.12157/) (2022)
- [An autonomous agent for auditing and improving the reliability of clinical AI models](https://mlsanity.com/paper/2507.05755/) (2025)

## Abstract (arXiv)

Self-supervision has demonstrated to be an effective learning strategy when training target tasks on small annotated data-sets. While current research focuses on creating novel pretext tasks to learn meaningful and reusable representations for the target task, these efforts obtain marginal performance gains compared to fully-supervised learning. Meanwhile, little attention has been given to study the robustness of networks trained in a self-supervised manner. In this work, we demonstrate that networks trained via self-supervised learning have superior robustness and generalizability compared to fully-supervised learning in the context of medical imaging. Our experiments on pneumonia detection in X-rays and multi-organ segmentation in CT yield consistent results exposing the hidden benefits of self-supervision for learning robust feature representations.