mlsanityData to 7 Oct 2026

Navigating the Shortcut Maze: A Comprehensive Analysis of Shortcut Learning in Text Classification by Language Models

arXiv 2409.17455 ↗First posted 26 Sep 2024cs.CL cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Language models (LMs), despite their advances, often depend on spurious correlations, undermining their accuracy and generalizability. This study addresses the overlooked impact of subtler, more complex shortcuts that compromise model reliability beyond oversimplified shortcuts. We introduce a comprehensive benchmark that categorizes shortcuts into occurrence, style, and concept, aiming to explore the nuanced ways in which these shortcuts influence the performance of LMs. Through extensive experiments across traditional LMs, large language models, and state-of-the-art robust models, our research systematically investigates models' resilience and susceptibilities to sophisticated shortcuts. Our benchmark and code can be found at: https://github.com/yuqing-zhou/shortcut-learning-in-text-classification.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 36 of its 45 references that are in the map. First posted on arXiv 26 Sep 2024.

Builds on

36 of 45 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  8. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  9. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  10. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  11. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  12. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
24 more
  1. 2020GoEmotions: A Dataset of Fine-Grained Emotionscited by 479
  2. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 337
  3. 2018Techniques for Interpretable Machine Learningcited by 237
  4. 2019Probing Neural Network Comprehension of Natural Language Argumentscited by 212
  5. 2020Invariant Rationalizationcited by 131
  6. 2022Spread Spurious Attribute: Improving Worst-group Accuracy with Spurious Attribute Estimationcited by 88
  7. 2021Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Modelscited by 78
  8. 2021Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Modelscited by 76
  9. 2023Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learningcited by 69
  10. 2023Simple and Fast Group Robustness by Automatic Feature Reweightingcited by 68
  11. 2020Robustness to Spurious Correlations in Text Classification via Automatically Generated Counterfactualscited by 67
  12. 2018Did the Model Understand the Question?cited by 60
  13. 2020Identifying Spurious Correlations for Robust Text Classificationcited by 59
  14. 2018Deriving Machine Attention from Human Rationalescited by 54
  15. 2018Explicit Bias Discovery in Visual Question Answering Modelscited by 42
  16. 2023Spawrious: A Benchmark for Fine Control of Spurious Correlation Biasescited by 41
  17. 2023Robust Learning with Progressive Data Expansion Against Spurious Correlationcited by 39
  18. 2021Understanding Interlocking Dynamics of Cooperative Rationalizationcited by 36
  19. 2019What does BERT Learn from Multiple-Choice Reading Comprehension Datasets?cited by 30
  20. 2022Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQAcited by 30
  21. 2023Towards Trustworthy Explanation: On Causal Rationalizationcited by 19
  22. 2020A Unified Taylor Framework for Revisiting Attribution Methodscited by 13
  23. 2022Less Learn Shortcut: Analyzing and Mitigating Learning of Spurious Feature-Label Correlationcited by 13
  24. 2023Understanding and Mitigating Spurious Correlations in Text Classification with Neighborhood Analysiscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 3.

10 papers.

  1. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  2. 2024Shortcut Learning in In-Context Learning: A Surveycited by 8
  3. 2025Self-Regularization with Sparse Autoencoders for Controllable LLM-based Classificationcited by 4
  4. 2025Language Models, Graph Searching, and Supervision Adulteration: When More Supervision is Less and How to Make More Morecited by 3
  5. 2024In-context Learning in Presence of Spurious Correlationscited by 2
  6. 2025LoRA Users Beware: A Few Spurious Tokens Can Manipulate Your Finetuned Modelcited by 2
  7. 2025Large Language Models Are Still Misled by Simple Bias Ensemblescited by 0
  8. 2026Confusion-Aware In-Context-Learning for Vision-Language Models in Robotic Manipulationcited by 0
  9. 2026Models Know Their Shortcuts: Deployment-Time Shortcut Mitigationcited by 0
  10. 2026Incomplete Prompt Jailbreaks in Large Language Modelscited by 0