# Augment like there's no tomorrow: Consistently performing neural networks for medical imaging

arXiv: https://arxiv.org/abs/2206.15274 · HTML: https://mlsanity.com/paper/2206.15274/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 55 references that are in the map.
- First posted on arXiv 30 Jun 2022.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks](https://mlsanity.com/paper/2102.11600/) (2021)
- [Gradient Starvation: A Learning Proclivity in Neural Networks](https://mlsanity.com/paper/2011.09468/) (2020)
- [TrivialAugment: Tuning-free Yet State-of-the-Art Data Augmentation](https://mlsanity.com/paper/2103.10158/) (2021)
- [Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology](https://mlsanity.com/paper/1902.06543/) (2019)
- [Revisiting ResNets: Improved Training and Scaling Strategies](https://mlsanity.com/paper/2103.07579/) (2021)
- [Whole-Slide Mitosis Detection in H&E Breast Histology Using PHH3 as a Reference to Train Distilled Stain-Invariant Convolutional Networks](https://mlsanity.com/paper/1808.05896/) (2018)
- [UniformAugment: A Search-free Probabilistic Data Augmentation Approach](https://mlsanity.com/paper/2003.14348/) (2020)
- [Development and Validation of a Deep Learning Algorithm for Improving Gleason Scoring of Prostate Cancer](https://mlsanity.com/paper/1811.06497/) (2018)
- [Neural Stain-Style Transfer Learning using GAN for Histopathological Images](https://mlsanity.com/paper/1710.08543/) (2017)
- [Whole-Slide Image Focus Quality: Automatic Assessment and Impact on AI Cancer Detection](https://mlsanity.com/paper/1901.04619/) (2019)
- [Spectral decoupling allows training transferable neural networks in medical imaging](https://mlsanity.com/paper/2103.17171/) (2021)

## Built on by (most cited first, 5 of 5)

- [MorphGen: Morphology-Guided Representation Learning for Robust Single-Domain Generalization in Histopathological Cancer Classification](https://mlsanity.com/paper/2509.00311/) (2025)
- [ProtoMedX: Towards Explainable Multi-Modal Prototype Learning for Bone Health Classification](https://mlsanity.com/paper/2509.14830/) (2025)
- [Are nuclear masks all you need for improved out-of-domain generalisation? A closer look at cancer classification in histopathology](https://mlsanity.com/paper/2411.09373/) (2024)
- [HistoEncoder: a digital pathology foundation model for prostate cancer](https://mlsanity.com/paper/2411.11458/) (2024)
- [CLEAR: Causal Learning Framework For Robust Histopathology Tumor Detection Under Out-Of-Distribution Shifts](https://mlsanity.com/paper/2510.14273/) (2025)

## Abstract (arXiv)

Deep neural networks have achieved impressive performance in a wide variety of medical imaging tasks. However, these models often fail on data not used during training, such as data originating from a different medical centre. How to recognize models suffering from this fragility, and how to design robust models are the main obstacles to clinical adoption. Here, we present general methods to identify causes for model generalisation failures and how to circumvent them. First, we use textitdistribution-shifted datasets to show that models trained with current state-of-the-art methods are highly fragile to variability encountered in clinical practice, and then develop a textitstrong augmentation strategy to address this fragility. Distribution-shifted datasets allow us to discover this fragility, which can otherwise remain undetected after validation against multiple external datasets. Strong augmentation allows us to train robust models achieving consistent performance under shifts from the training data distribution. Importantly, we demonstrate that strong augmentation yields biomedical imaging models which retain high performance when applied to real-world clinical data. Our results pave the way for the development and evaluation of reliable and robust neural networks in clinical practice.