# Mask of truth: model sensitivity to unexpected regions of medical images

arXiv: https://arxiv.org/abs/2412.04030 · HTML: https://mlsanity.com/paper/2412.04030/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 71 references that are in the map.
- First posted on arXiv 5 Dec 2024.

## Builds on (most cited first, 30 of 40)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [PadChest: A large chest x-ray image dataset with multi-label annotated reports](https://mlsanity.com/paper/1901.07441/) (2019)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imaging](https://mlsanity.com/paper/1909.12475/) (2019)
- [Metrics reloaded: Recommendations for image analysis validation](https://mlsanity.com/paper/2206.01653/) (2022)
- [Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data](https://mlsanity.com/paper/2308.02463/) (2023)
- [Impossibility Theorems for Feature Attribution](https://mlsanity.com/paper/2212.11870/) (2022)
- [Fast Diffusion-Based Counterfactuals for Shortcut Removal and Generation](https://mlsanity.com/paper/2312.14223/) (2023)
- [Are Vision Transformers Robust to Spurious Correlations?](https://mlsanity.com/paper/2203.09125/) (2022)
- [Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth](https://mlsanity.com/paper/1702.03407/) (2017)
- [AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge](https://mlsanity.com/paper/2302.01738/) (2023)
- [CheXmask: a large-scale dataset of anatomical segmentation masks for multi-center chest x-ray images](https://mlsanity.com/paper/2307.03293/) (2023)
- [(De)Constructing Bias on Skin Lesion Datasets](https://mlsanity.com/paper/1904.08818/) (2019)
- [RadEdit: stress-testing biomedical vision models via diffusion image editing](https://mlsanity.com/paper/2312.12865/) (2023)
- [Detecting Shortcuts in Medical Images -- A Case Study in Chest X-rays](https://mlsanity.com/paper/2211.04279/) (2022)
- [VisionUnite: A Vision-Language Foundation Model for Ophthalmology Enhanced with Clinical Knowledge](https://mlsanity.com/paper/2408.02865/) (2024)
- [Advancing human-centric AI for robust X-ray analysis through holistic self-supervised learning](https://mlsanity.com/paper/2405.01469/) (2024)
- [Right for the Wrong Reason: Can Interpretable ML Techniques Detect Spurious Correlations?](https://mlsanity.com/paper/2307.12344/) (2023)
- [Shortcut Learning in Medical Image Segmentation](https://mlsanity.com/paper/2403.06748/) (2024)
- [Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics](https://mlsanity.com/paper/2409.16756/) (2024)
- [Navigating the landscape of multimodal AI in medicine: a scoping review on technical challenges and clinical applications](https://mlsanity.com/paper/2411.03782/) (2024)
- [Counterfactual contrastive learning: robust representations via causal image synthesis](https://mlsanity.com/paper/2403.09605/) (2024)
- [Source Matters: Source Dataset Impact on Model Robustness in Medical Imaging](https://mlsanity.com/paper/2403.04484/) (2024)

## Built on by (most cited first, 6 of 6)

- [CheXmask-U: Quantifying uncertainty in landmark-based anatomical segmentation for X-ray images](https://mlsanity.com/paper/2512.10715/) (2025)
- [Fairness and Robustness of CLIP-Based Models for Chest X-rays](https://mlsanity.com/paper/2507.21291/) (2025)
- [Dataset Diversity Metrics and Impact on Classification Models](https://mlsanity.com/paper/2603.15276/) (2026)
- [The Impact of Preprocessing Methods on Racial Encoding and Model Robustness in CXR Diagnosis](https://mlsanity.com/paper/2603.05157/) (2026)
- [Symbal: Detecting Systematic Misalignments in Model-Generated Captions](https://mlsanity.com/paper/2607.15216/) (2026)
- [Look What the Probes Dragged In! Real-World Chest X-ray Shortcuts in MedCLIP](https://mlsanity.com/paper/2608.12086/) (2026)

## Abstract (arXiv)

The development of larger models for medical image analysis has led to increased performance. However, it also affected our ability to explain and validate model decisions. Models can use non-relevant parts of images, also called spurious correlations or shortcuts, to obtain high performance on benchmark datasets but fail in real-world scenarios. In this work, we challenge the capacity of convolutional neural networks (CNN) to classify chest X-rays and eye fundus images while masking out clinically relevant parts of the image. We show that all models trained on the PadChest dataset, irrespective of the masking strategy, are able to obtain an Area Under the Curve (AUC) above random. Moreover, the models trained on full images obtain good performance on images without the region of interest (ROI), even superior to the one obtained on images only containing the ROI. We also reveal a possible spurious correlation in the Chaksu dataset while the performances are more aligned with the expectation of an unbiased model. We go beyond the performance analysis with the usage of the explainability method SHAP and the analysis of embeddings. We asked a radiology resident to interpret chest X-rays under different masking to complement our findings with clinical knowledge. Our code is available at https://github.com/TheoSourget/MMC_Masking and https://github.com/TheoSourget/MMC_Masking_EyeFundus