# Detecting Shortcut Learning for Fair Medical AI using Shortcut Testing

arXiv: https://arxiv.org/abs/2207.10384 · HTML: https://mlsanity.com/paper/2207.10384/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 66 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 30 of 31)

- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [A deep learning system for differential diagnosis of skin diseases](https://mlsanity.com/paper/1909.05382/) (2019)
- [Re-imagining Algorithmic Fairness in India and Beyond](https://mlsanity.com/paper/2101.09995/) (2021)
- [Estimating Example Difficulty Using Variance of Gradients](https://mlsanity.com/paper/2008.11600/) (2020)
- [Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models](https://mlsanity.com/paper/2103.06922/) (2021)
- [Achieving Fairness through Adversarial Learning: an Application to Recidivism Prediction](https://mlsanity.com/paper/1807.00199/) (2018)
- [Predicting Cardiovascular Risk Factors from Retinal Fundus Photographs using Deep Learning](https://mlsanity.com/paper/1708.09843/) (2017)
- [Causally motivated Shortcut Removal Using Auxiliary Labels](https://mlsanity.com/paper/2105.06422/) (2021)
- [Improving the Fairness of Chest X-ray Classifiers](https://mlsanity.com/paper/2203.12609/) (2022)
- [Diagnosing failures of fairness transfer across distribution shift in real-world medical settings](https://mlsanity.com/paper/2202.01034/) (2022)
- [Automatic Shortcut Removal for Self-Supervised Representation Learning](https://mlsanity.com/paper/2002.08822/) (2020)
- [Blind Justice: Fairness with Encrypted Sensitive Attributes](https://mlsanity.com/paper/1806.03281/) (2018)
- [Deep Learning Applied to Chest X-Rays: Exploiting and Preventing Shortcuts](https://mlsanity.com/paper/2009.10132/) (2020)
- [This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technology](https://mlsanity.com/paper/1909.11869/) (2019)
- [Write It Like You See It: Detectable Differences in Clinical Notes By Race Lead To Differential Model Recommendations](https://mlsanity.com/paper/2205.03931/) (2022)
- [A Near-Optimal Algorithm for Debiasing Trained Machine Learning Models](https://mlsanity.com/paper/2106.12887/) (2021)
- [Removing biased data to improve fairness and accuracy](https://mlsanity.com/paper/2102.03054/) (2021)
- [Joint Optimization of AI Fairness and Utility: A Human-Centered Approach](https://mlsanity.com/paper/2002.01621/) (2020)
- [Does "AI" stand for augmenting inequality in the era of covid-19 healthcare?](https://mlsanity.com/paper/2105.07844/) (2021)
- [Model-based metrics: Sample-efficient estimates of predictive model subpopulation performance](https://mlsanity.com/paper/2104.12231/) (2021)

## Built on by (most cited first, 30 of 52)

- [Fast Diffusion-Based Counterfactuals for Shortcut Removal and Generation](https://mlsanity.com/paper/2312.14223/) (2023)
- [Addressing Fairness Issues in Deep Learning-Based Medical Image Analysis: A Systematic Review](https://mlsanity.com/paper/2209.13177/) (2022)
- [Shortcut Learning in Medical Image Segmentation](https://mlsanity.com/paper/2403.06748/) (2024)
- [Slicing Through Bias: Explaining Performance Gaps in Medical Image Analysis using Slice Discovery Methods](https://mlsanity.com/paper/2406.12142/) (2024)
- [BiasPruner: Debiased Continual Learning for Medical Image Classification](https://mlsanity.com/paper/2407.08609/) (2024)
- [Mind the Graph When Balancing Data for Fairness or Robustness](https://mlsanity.com/paper/2406.17433/) (2024)
- [A Systematic Review of Generalization Research in Medical Image Classification](https://mlsanity.com/paper/2403.12167/) (2024)
- [Are demographically invariant models and representations in medical imaging fair?](https://mlsanity.com/paper/2305.01397/) (2023)
- [Improving Intervention Efficacy via Concept Realignment in Concept Bottleneck Models](https://mlsanity.com/paper/2405.01531/) (2024)
- [Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Models](https://mlsanity.com/paper/2505.10930/) (2025)
- [Manifestations of Xenophobia in AI Systems](https://mlsanity.com/paper/2212.07877/) (2022)
- [Towards objective and systematic evaluation of bias in artificial intelligence for medical imaging](https://mlsanity.com/paper/2311.02115/) (2023)
- [Shortcut Learning Susceptibility in Vision Classifiers](https://mlsanity.com/paper/2502.09150/) (2025)
- [Gaze-Guided Learning: Avoiding Shortcut Bias in Visual Classification](https://mlsanity.com/paper/2504.05583/) (2025)
- [Evaluating Model Bias Requires Characterizing its Mistakes](https://mlsanity.com/paper/2407.10633/) (2024)
- [Clever Materials: When Models Identify Good Materials for the Wrong Reasons](https://mlsanity.com/paper/2602.17730/) (2026)
- [Predicting risk of cardiovascular disease using retinal OCT imaging](https://mlsanity.com/paper/2403.18873/) (2024)
- [Fairness Evolution in Continual Learning for Medical Imaging](https://mlsanity.com/paper/2406.02480/) (2024)
- [Positive-Sum Fairness: Leveraging Demographic Attributes to Achieve Fair AI Outcomes Without Sacrificing Group Gains](https://mlsanity.com/paper/2409.19940/) (2024)
- [Ensuring Medical AI Safety: Interpretability-Driven Detection and Mitigation of Spurious Model Behavior and Associated Data](https://mlsanity.com/paper/2501.13818/) (2025)
- [Spurious Correlations and Beyond: Understanding and Mitigating Shortcut Learning in SDOH Extraction with Large Language Models](https://mlsanity.com/paper/2506.00134/) (2025)
- [Feature salience - not task-informativeness - drives machine learning model explanations](https://mlsanity.com/paper/2602.09238/) (2026)
- [Learning beyond Site Bias for OOD Generalization in Brain Networks](https://mlsanity.com/paper/2605.06050/) (2026)
- [Common-Sense Bias Modeling for Classification Tasks](https://mlsanity.com/paper/2401.13213/) (2024)
- [Clinical Domain Knowledge-Derived Template Improves Post Hoc AI Explanations in Pneumothorax Classification](https://mlsanity.com/paper/2403.18871/) (2024)
- [Machine Learning Applications to Diffuse Reflectance Spectroscopy in Optical Diagnosis; A Systematic Review](https://mlsanity.com/paper/2503.02905/) (2025)
- [MOSCARD -- Causal Reasoning and De-confounding for Multimodal Opportunistic Screening of Cardiovascular Adverse Events](https://mlsanity.com/paper/2506.19174/) (2025)
- [Correct-By-Construction: Certified Individual Fairness through Neural Network Training](https://mlsanity.com/paper/2508.15642/) (2025)
- [RoentMod: A Synthetic Chest X-Ray Modification Model to Identify and Correct Image Interpretation Model Shortcuts](https://mlsanity.com/paper/2509.08640/) (2025)
- [Did Models Learn Sufficiently? Attribution-Guided Training via Subset-Selected Counterfactual Augmentation](https://mlsanity.com/paper/2511.12100/) (2025)

## Abstract (arXiv)

Machine learning (ML) holds great promise for improving healthcare, but it is critical to ensure that its use will not propagate or amplify health disparities. An important step is to characterize the (un)fairness of ML models - their tendency to perform differently across subgroups of the population - and to understand its underlying mechanisms. One potential driver of algorithmic unfairness, shortcut learning, arises when ML models base predictions on improper correlations in the training data. However, diagnosing this phenomenon is difficult, especially when sensitive attributes are causally linked with disease. Using multi-task learning, we propose the first method to assess and mitigate shortcut learning as a part of the fairness assessment of clinical ML systems, and demonstrate its application to clinical tasks in radiology and dermatology. Finally, our approach reveals instances when shortcutting is not responsible for unfairness, highlighting the need for a holistic approach to fairness mitigation in medical AI.