# The Clever Hans Mirage: A Comprehensive Survey on Spurious Correlations in Machine Learning

arXiv: https://arxiv.org/abs/2402.12715 · HTML: https://mlsanity.com/paper/2402.12715/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 111 of its 203 references that are in the map.
- First posted on arXiv 20 Feb 2024.

## Builds on (most cited first, 30 of 111)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Invariant Risk Minimization](https://mlsanity.com/paper/1907.02893/) (2019)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization](https://mlsanity.com/paper/1911.08731/) (2019)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [In Search of Lost Domain Generalization](https://mlsanity.com/paper/2007.01434/) (2020)
- [Towards Understanding Sycophancy in Language Models](https://mlsanity.com/paper/2310.13548/) (2023)
- [Deeper, Broader and Artier Domain Generalization](https://mlsanity.com/paper/1710.03077/) (2017)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Generalizing to Unseen Domains: A Survey on Domain Generalization](https://mlsanity.com/paper/2103.03097/) (2021)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Just Train Twice: Improving Group Robustness without Training Group Information](https://mlsanity.com/paper/2107.09044/) (2021)
- [Measuring Robustness to Natural Distribution Shifts in Image Classification](https://mlsanity.com/paper/2007.00644/) (2020)
- [Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification](https://mlsanity.com/paper/1903.04561/) (2019)
- [Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance](https://mlsanity.com/paper/2006.14779/) (2020)
- [QuAC : Question Answering in Context](https://mlsanity.com/paper/1808.07036/) (2018)
- [Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations](https://mlsanity.com/paper/2204.02937/) (2022)

## Built on by (most cited first, 30 of 41)

- [Shortcut Learning in Generalist Robot Policies: The Role of Dataset Diversity and Fragmentation](https://mlsanity.com/paper/2508.06426/) (2025)
- [Navigating Shortcuts, Spurious Correlations, and Confounders: From Origins via Detection to Mitigation](https://mlsanity.com/paper/2412.05152/) (2024)
- [MOS: Modeling Object-Scene Associations in Generalized Category Discovery](https://mlsanity.com/paper/2503.12035/) (2025)
- [Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization](https://mlsanity.com/paper/2502.01347/) (2025)
- [Policy Contrastive Decoding for Robotic Foundation Models](https://mlsanity.com/paper/2505.13255/) (2025)
- [Evaluating the Logical Reasoning Abilities of Large Reasoning Models](https://mlsanity.com/paper/2505.11854/) (2025)
- [Focus On This, Not That! Steering LLMs with Adaptive Feature Specification](https://mlsanity.com/paper/2410.22944/) (2024)
- [Mitigating Spurious Correlations in LLMs via Causality-Aware Post-Training](https://mlsanity.com/paper/2506.09433/) (2025)
- [The Artificial Intelligence Cognitive Examination: A Survey on the Evolution of Multimodal Evaluation from Recognition to Reasoning](https://mlsanity.com/paper/2510.04141/) (2025)
- [Do regularization methods for shortcut mitigation work as intended?](https://mlsanity.com/paper/2503.17015/) (2025)
- [BackMix: Regularizing Open Set Recognition by Removing Underlying Fore-Background Priors](https://mlsanity.com/paper/2503.17717/) (2025)
- [Towards Utilising a Range of Neural Activations for Comprehending Representational Associations](https://mlsanity.com/paper/2411.10019/) (2024)
- [Evaluating and Improving Cultural Awareness of Reward Models for LLM Alignment](https://mlsanity.com/paper/2509.21798/) (2025)
- [SEA-Guard: Culturally Grounded Multilingual Safeguard for Southeast Asia](https://mlsanity.com/paper/2602.01618/) (2026)
- [DINOv2 Rocks Geological Image Analysis: Classification, Segmentation, and Interpretability](https://mlsanity.com/paper/2407.18100/) (2024)
- [Enhancing Event Reasoning in Large Language Models through Instruction Fine-Tuning with Semantic Causal Graphs](https://mlsanity.com/paper/2409.00209/) (2024)
- [FairDropout: Using Example-Tied Dropout to Enhance Generalization of Minority Groups](https://mlsanity.com/paper/2502.06695/) (2025)
- [Learning to Substitute Components for Compositional Generalization](https://mlsanity.com/paper/2502.20834/) (2025)
- [CLARiTy: A Vision Transformer for Multi-Label Classification and Weakly-Supervised Localization of Chest X-ray Pathologies](https://mlsanity.com/paper/2512.16700/) (2025)
- [GIM: Evaluating models via tasks that integrate multiple cognitive domains](https://mlsanity.com/paper/2605.18663/) (2026)
- [Autoencoder based approach for the mitigation of spurious correlations](https://mlsanity.com/paper/2406.18901/) (2024)
- [Spurious Correlations in Concept Drift: Can Explanatory Interaction Help?](https://mlsanity.com/paper/2407.16515/) (2024)
- [Improving Adversarial Robustness in Android Malware Detection by Reducing the Impact of Spurious Correlations](https://mlsanity.com/paper/2408.16025/) (2024)
- [Focusing Image Generation to Mitigate Spurious Correlations](https://mlsanity.com/paper/2412.19457/) (2024)
- [Recent Advances, Applications and Open Challenges in Machine Learning for Health: Reflections from Research Roundtables at ML4H 2024 Symposium](https://mlsanity.com/paper/2502.06693/) (2025)
- [Active Data Sampling and Generation for Bias Remediation](https://mlsanity.com/paper/2503.20414/) (2025)
- [Training on Plausible Counterfactuals Removes Spurious Correlations](https://mlsanity.com/paper/2505.16583/) (2025)
- [What is Adversarial Training for Diffusion Models?](https://mlsanity.com/paper/2505.21742/) (2025)
- [Model Guidance via Robust Feature Attribution](https://mlsanity.com/paper/2506.19680/) (2025)
- [BM-CL: Bias Mitigation through the lens of Continual Learning](https://mlsanity.com/paper/2509.01730/) (2025)

## Abstract (arXiv)

Back in the early 20th century, a horse named Hans appeared to perform arithmetic and other intellectual tasks during exhibitions in Germany, while it actually relied solely on involuntary cues in the body language from the human trainer. Modern machine learning models are no different. These models are known to be sensitive to spurious correlations between non-essential features of the inputs (e.g., background, texture, and secondary objects) and the corresponding labels. Such features and their correlations with the labels are known as "spurious" because they tend to change with shifts in real-world data distributions, which can negatively impact the model's generalization and robustness. In this paper, we provide a comprehensive survey of this emerging issue, along with a fine-grained taxonomy of existing state-of-the-art methods for addressing spurious correlations in machine learning models. Additionally, we summarize existing datasets, benchmarks, and metrics to facilitate future research. The paper concludes with a discussion of the broader impacts, the recent advancements, and future challenges in the era of generative AI, aiming to provide valuable insights for researchers in the related domains of the machine learning community.