mlsanityData to 14 Oct 2026

Shortcut Learning in Deep Neural Networks

arXiv 2004.07780 ↗First posted 16 Apr 2020cs.CV cs.AI cs.LGcited by 1,969

Not on the map. Why?

Cited by 88 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep learning has triggered the current rise of artificial intelligence and is the workhorse of today's machine intelligence. Numerous success stories have rapidly spread all over science, industry and society, but its limitations have only recently come into focus. In this perspective we seek to distill how many of deep learning's problems can be seen as different symptoms of the same underlying problem: shortcut learning. Shortcuts are decision rules that perform well on standard benchmarks but fail to transfer to more challenging testing conditions, such as real-world scenarios. Related issues are known in Comparative Psychology, Education and Linguistics, suggesting that shortcut learning may be a common characteristic of learning systems, biological and artificial alike. Based on these observations, we develop a set of recommendations for model interpretation and benchmarking, highlighting recent advances in machine learning to improve robustness and transferability from the lab to real-world applications.

Key facts. Cited by 1,969 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 155; 2022: 267; 2023: 320; 2024: 285; 2025: 336; 2026: 605.

1,969 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  3. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  4. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  5. 2021Domain Generalization: A Surveycited by 764
  6. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 610
  7. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 561
  8. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  9. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 490
  10. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 411
  11. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 381
  12. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 312
38 more
  1. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  2. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  3. 2022Transformers Learn Shortcuts to Automatacited by 268
  4. 2021Invariance Principle Meets Information Bottleneck for Out-of-Distribution Generalizationcited by 220
  5. 2025Video models are zero-shot learners and reasonerscited by 220
  6. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 202
  7. 2022Metrics reloaded: Recommendations for image analysis validationcited by 195
  8. 2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
  9. 2021Partial success in closing the gap between human and machine visioncited by 185
  10. 2021Simple data balancing achieves competitive worst-group-accuracycited by 178
  11. 2024Sycophancy to Subterfuge: Investigating Reward-Tampering in Large Language Modelscited by 178
  12. 2023Clever Hans or Neural Theory of Mind? Stress Testing Social Reasoning in Large Language Modelscited by 163
  13. 2025Do generative video models understand physical principles?cited by 161
  14. 2021Physion: Evaluating Physical Prediction from Vision in Humans and Machinescited by 153
  15. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 152
  16. 2022On Feature Learning in the Presence of Spurious Correlationscited by 145
  17. 2022Learning Causally Invariant Representations for Out-of-Distribution Generalization on Graphscited by 144
  18. 2022Interventional Causal Representation Learningcited by 142
  19. 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
  20. 2023The Science of Detecting LLM-Generated Textscited by 128
  21. 2023Large Language Models as Zero-Shot Conversational Recommenderscited by 127
  22. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  23. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 125
  24. 2021Goal Misgeneralization in Deep Reinforcement Learningcited by 123
  25. 2021Causality-inspired Single-source Domain Generalization for Medical Image Segmentationcited by 119
  26. 20223D Common Corruptions and Data Augmentationcited by 119
  27. 2021Interpretable Deep Learning: Interpretation, Interpretability, Trustworthiness, and Beyondcited by 118
  28. 2021Gender Bias in Machine Translationcited by 117
  29. 2021Learning Debiased Representation via Disentangled Feature Augmentationcited by 116
  30. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  31. 2022Algorithms to estimate Shapley value feature attributionscited by 115
  32. 2021Can contrastive learning avoid shortcut solutions?cited by 114
  33. 2022Disentangled Representation Learningcited by 112
  34. 2021Counterfactual Generative Networkscited by 111
  35. 2021Causal Attention for Interpretable and Generalizable Graph Classificationcited by 108
  36. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  37. 2024Entropy is not Enough for Test-Time Adaptation: From the Perspective of Disentangled Factorscited by 106
  38. 2021Abstraction and Analogy-Making in Artificial Intelligencecited by 103

50 most cited of 1,969 All 1,969 (JSON)