# Shortcut Learning in Medical Image Segmentation

arXiv: https://arxiv.org/abs/2403.06748 · HTML: https://mlsanity.com/paper/2403.06748/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations](https://mlsanity.com/paper/1707.03237/) (2017)
- [Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1710.05006/) (2017)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Skin Lesion Analysis toward Melanoma Detection: A Challenge at the International Symposium on Biomedical Imaging (ISBI) 2016, hosted by the International Skin Imaging Collaboration (ISIC)](https://mlsanity.com/paper/1605.01397/) (2016)
- [On Feature Learning in the Presence of Spurious Correlations](https://mlsanity.com/paper/2210.11369/) (2022)
- [Detecting Shortcut Learning for Fair Medical AI using Shortcut Testing](https://mlsanity.com/paper/2207.10384/) (2022)
- [Machine Learning with Multi-Site Imaging Data: An Empirical Study on the Impact of Scanner Effects](https://mlsanity.com/paper/1910.04597/) (2019)
- [Don't blame Dataset Shift! Shortcut Learning due to Gradients and Cross Entropy](https://mlsanity.com/paper/2308.12553/) (2023)
- [What do neural networks learn in image classification? A frequency shortcut perspective](https://mlsanity.com/paper/2307.09829/) (2023)
- [A Survey on Deep Learning for Skin Lesion Segmentation](https://mlsanity.com/paper/2206.00356/) (2022)
- [Detecting Shortcuts in Medical Images -- A Case Study in Chest X-rays](https://mlsanity.com/paper/2211.04279/) (2022)
- [AutoFB: Automating Fetal Biometry Estimation from Standard Ultrasound Planes](https://mlsanity.com/paper/2107.05255/) (2021)
- [PyMIC: A deep learning toolkit for annotation-efficient medical image segmentation](https://mlsanity.com/paper/2208.09350/) (2022)
- [DTU-Net: Learning Topological Similarity for Curvilinear Structure Segmentation](https://mlsanity.com/paper/2205.11115/) (2022)
- [Removing confounding information from fetal ultrasound images](https://mlsanity.com/paper/2303.13918/) (2023)

## Built on by (most cited first, 16 of 16)

- [AbdomenAtlas: A Large-Scale, Detailed-Annotated, & Multi-Center Dataset for Efficient Transfer Learning and Open Algorithmic Benchmarking](https://mlsanity.com/paper/2407.16697/) (2024)
- [Touchstone Benchmark: Are We on the Right Way for Evaluating AI Algorithms for Medical Segmentation?](https://mlsanity.com/paper/2411.03670/) (2024)
- [FreqDebias: Towards Generalizable Deepfake Detection via Consistency-Driven Frequency Debiasing](https://mlsanity.com/paper/2509.22412/) (2025)
- [Mask of truth: model sensitivity to unexpected regions of medical images](https://mlsanity.com/paper/2412.04030/) (2024)
- [Shortcut Learning Susceptibility in Vision Classifiers](https://mlsanity.com/paper/2502.09150/) (2025)
- [Seeing Isn't Always Believing: Analysis of Grad-CAM Faithfulness and Localization Reliability in Lung Cancer CT Classification](https://mlsanity.com/paper/2601.12826/) (2026)
- [A Contrast-Agnostic Method for Ultra-High Resolution Claustrum Segmentation](https://mlsanity.com/paper/2411.15388/) (2024)
- [DiSSECT: Structuring Transfer-Ready Medical Image Representations through Discrete Self-Supervision](https://mlsanity.com/paper/2509.18765/) (2025)
- [Weight Space Correlation Analysis: Quantifying Feature Utilization in Deep Learning Models](https://mlsanity.com/paper/2512.13144/) (2025)
- [StealthMark: Harmless and Stealthy Ownership Verification for Medical Segmentation via Uncertainty-Guided Backdoors](https://mlsanity.com/paper/2601.17107/) (2026)
- [Right Regions, Wrong Labels: Semantic Label Flips in Segmentation under Correlation Shift](https://mlsanity.com/paper/2604.13326/) (2026)
- [idSCD: Identifying Training Datasets through Semantic Correlation Descriptors](https://mlsanity.com/paper/2605.30462/) (2026)
- [ReMoDEx: A Local-to-Global Relevance-Based Model Decision Explainability Framework for large-Scale Image Datasets](https://mlsanity.com/paper/2607.06889/) (2026)
- [Towards Whole-Study Screening for Congenital Heart Disease in Fetal Ultrasound Using Multiple Instance Learning](https://mlsanity.com/paper/2609.31376/) (2026)
- [Is invariance all you need for algorithmic fairness? Removing demographic information can create new bias](https://mlsanity.com/paper/2609.32004/) (2026)
- [Prospective Prediction of OOD Degradation from Source-Side Training Dynamics](https://mlsanity.com/paper/2610.12397/) (2026)

## Abstract (arXiv)

Shortcut learning is a phenomenon where machine learning models prioritize learning simple, potentially misleading cues from data that do not generalize well beyond the training set. While existing research primarily investigates this in the realm of image classification, this study extends the exploration of shortcut learning into medical image segmentation. We demonstrate that clinical annotations such as calipers, and the combination of zero-padded convolutions and center-cropped training sets in the dataset can inadvertently serve as shortcuts, impacting segmentation accuracy. We identify and evaluate the shortcut learning on two different but common medical image segmentation tasks. In addition, we suggest strategies to mitigate the influence of shortcut learning and improve the generalizability of the segmentation models. By uncovering the presence and implications of shortcuts in medical image segmentation, we provide insights and methodologies for evaluating and overcoming this pervasive challenge and call for attention in the community for shortcuts in segmentation. Our code is public at https://github.com/nina-weng/shortcut_skinseg .