# MedMNIST-C: Comprehensive benchmark and improved classifier robustness by simulating realistic image corruptions

arXiv: https://arxiv.org/abs/2406.17536 · HTML: https://mlsanity.com/paper/2406.17536/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 34 references that are in the map.
- First posted on arXiv 25 Jun 2024.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Vision Transformers are Robust Learners](https://mlsanity.com/paper/2105.07581/) (2021)
- [MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis](https://mlsanity.com/paper/2010.14925/) (2020)
- [A Systematic Review of Robustness in Deep Learning for Computer Vision: Mind the gap?](https://mlsanity.com/paper/2112.00639/) (2021)
- [Benchmarking the Robustness of Deep Neural Networks to Common Corruptions in Digital Pathology](https://mlsanity.com/paper/2206.14973/) (2022)
- [DRvertGRADUATE: uncertainty-aware deep learning-based diabetic retinopathy grading in eye fundus images](https://mlsanity.com/paper/1910.11777/) (2019)
- [Out-of-Domain Robustness via Targeted Augmentations](https://mlsanity.com/paper/2302.11861/) (2023)
- [Evaluating the Robustness of Self-Supervised Learning in Medical Imaging](https://mlsanity.com/paper/2105.06986/) (2021)
- [Assessing and Enhancing Robustness of Deep Learning Models with Corruption Emulation in Digital Pathology](https://mlsanity.com/paper/2310.20427/) (2023)
- [Robustness Stress Testing in Medical Image Classification](https://mlsanity.com/paper/2308.06889/) (2023)

## Built on by (most cited first, 17 of 17)

- [On the Robustness of Medical Vision-Language Models: Are they Truly Generalizable?](https://mlsanity.com/paper/2505.15425/) (2025)
- [Rate-In: Information-Driven Adaptive Dropout Rates for Improved Inference-Time Uncertainty Estimation](https://mlsanity.com/paper/2412.07169/) (2024)
- [Medical Image Classification with KAN-Integrated Transformers and Dilated Neighborhood Attention](https://mlsanity.com/paper/2502.13693/) (2025)
- [Extending ZACH-ViT to Robust Medical Imaging: Corruption and Adversarial Stress Testing in Low-Data Regimes](https://mlsanity.com/paper/2604.06099/) (2026)
- [T3: Test-Time Model Merging in VLMs for Zero-Shot Medical Imaging Analysis](https://mlsanity.com/paper/2510.27265/) (2025)
- [Beyond Overconfidence: Foundation Models Redefine Calibration in Deep Neural Networks](https://mlsanity.com/paper/2506.09593/) (2025)
- [Stylizing ViT: Anatomy-Preserving Instance Style Transfer for Domain Generalization](https://mlsanity.com/paper/2601.17586/) (2026)
- [HypCBC: Domain-Invariant Hyperbolic Cross-Branch Consistency for Generalizable Medical Image Analysis](https://mlsanity.com/paper/2602.03264/) (2026)
- [An autonomous agent for auditing and improving the reliability of clinical AI models](https://mlsanity.com/paper/2507.05755/) (2025)
- [Deep Feedback Models](https://mlsanity.com/paper/2509.15905/) (2025)
- [LVLM-Aided Alignment of Task-Specific Vision Models](https://mlsanity.com/paper/2512.21985/) (2025)
- [MedQ-Deg: A Multidimensional Benchmark for Evaluating MLLMs Across Medical Image Quality Degradations](https://mlsanity.com/paper/2603.07769/) (2026)
- [Useful nonrobust features are ubiquitous in biomedical images](https://mlsanity.com/paper/2604.22579/) (2026)
- [MoPET: Parameter-Efficient Mixture-of-Experts for Unified Medical Image Classification](https://mlsanity.com/paper/2607.29462/) (2026)
- [Simple, Safe, and Overlooked: Reclaiming Sustainable Domain Generalization with Statistical Color Matching](https://mlsanity.com/paper/2608.18915/) (2026)
- [Looks the Same, Answers Differently: Flip-Direction Steering for Robust Vision-Language Reasoning](https://mlsanity.com/paper/2609.28851/) (2026)
- [Signal-Noise Factorization Isolates Nuisance Variation into Removable Subspaces](https://mlsanity.com/paper/2610.00751/) (2026)

## Abstract (arXiv)

The integration of neural-network-based systems into clinical practice is limited by challenges related to domain generalization and robustness. The computer vision community established benchmarks such as ImageNet-C as a fundamental prerequisite to measure progress towards those challenges. Similar datasets are largely absent in the medical imaging community which lacks a comprehensive benchmark that spans across imaging modalities and applications. To address this gap, we create and open-source MedMNIST-C, a benchmark dataset based on the MedMNIST+ collection covering 12 datasets and 9 imaging modalities. We simulate task and modality-specific image corruptions of varying severity to comprehensively evaluate the robustness of established algorithms against real-world artifacts and distribution shifts. We further provide quantitative evidence that our simple-to-use artificial corruptions allow for highly performant, lightweight data augmentation to enhance model robustness. Unlike traditional, generic augmentation strategies, our approach leverages domain knowledge, exhibiting significantly higher robustness when compared to widely adopted methods. By introducing MedMNIST-C and open-sourcing the corresponding library allowing for targeted data augmentations, we contribute to the development of increasingly robust methods tailored to the challenges of medical imaging. The code is available at https://github.com/francescodisalvo05/medmnistc-api .