mlsanityData to 7 Oct 2026

Towards Interpreting and Mitigating Shortcut Learning Behavior of NLU Models

arXiv 2103.06922 ↗First posted 11 Mar 2021cs.CL cs.LGcited by 78

Not on the map. Why?

Abstract (arXiv)

Recent studies indicate that NLU models are prone to rely on shortcut features for prediction, without achieving true language understanding. As a result, these models fail to generalize to real-world out-of-distribution data. In this work, we show that the words in the NLU training set can be modeled as a long-tailed distribution. There are two findings: 1) NLU models have strong preference for features located at the head of the long-tailed distribution, and 2) Shortcut features are picked up during very early few iterations of the model training. These two observations are further employed to formulate a measurement which can quantify the shortcut degree of each training sample. Based on this shortcut measurement, we propose a shortcut mitigation framework LTGR, to suppress the model from making overconfident predictions for samples with large shortcut degree. Experimental results on three NLU benchmarks demonstrate that our long-tailed distribution explanation accurately reflects the shortcut learning behavior of NLU models. Experimental analysis further indicates that LTGR can improve the generalization accuracy on OOD data, while preserving the accuracy on in-distribution data.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). Builds on 48 of its 52 references that are in the map. First posted on arXiv 11 Mar 2021.

Builds on

48 of 52 references.

References · 48

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  6. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  7. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  8. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  9. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  10. 2019When Does Label Smoothing Help?cited by 985
  11. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  12. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
36 more
  1. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  2. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  3. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  4. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  5. 2020The Pitfalls of Simplicity Bias in Neural Networkscited by 349
  6. 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 342
  7. 2017Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answeringcited by 337
  8. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  9. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  10. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  11. 2019Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biasescited by 279
  12. 2018Learning Not to Learn: Training Deep Neural Networks with Biased Datacited by 255
  13. 2018Techniques for Interpretable Machine Learningcited by 237
  14. 2017Embedding Watermarks into Deep Neural Networkscited by 227
  15. 2019Probing Neural Network Comprehension of Natural Language Argumentscited by 212
  16. 2019Learning De-biased Representations with Biased Representationscited by 199
  17. 2019RUBi: Reducing Unimodal Biases in Visual Question Answeringcited by 183
  18. 2019Towards Debiasing Fact Verification Modelscited by 143
  19. 2020An Empirical Study on Robustness to Spurious Correlations using Pre-trained Language Modelscited by 133
  20. 2019Taking a HINT: Leveraging Explanations to Make Vision and Language Models More Groundedcited by 127
  21. 2019End-to-End Bias Mitigation by Modelling Biases in Corporacited by 127
  22. 2019Learning Robust Representations by Projecting Superficial Statistics Outcited by 123
  23. 2019Unlearn Dataset Bias in Natural Language Inference by Fitting the Residualcited by 119
  24. 2020Towards Debiasing NLU Models from Unknown Biasescited by 116
  25. 2020An Embarrassingly Simple Approach for Trojan Attack in Deep Neural Networkscited by 92
  26. 2017Measuring the tendency of CNNs to Learn Surface Statistical Regularitiescited by 87
  27. 2020Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performancecited by 85
  28. 2019[Extended version] Rethinking Deep Neural Network Ownership Verification: Embedding Passports to Defeat Ambiguity Attackscited by 82
  29. 2018Did the Model Understand the Question?cited by 60
  30. 2018Knowledgeable Reader: Enhancing Cloze-Style Reading Comprehension with External Commonsense Knowledgecited by 48
  31. 2018Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledgecited by 45
  32. 2018Explicit Bias Discovery in Visual Question Answering Modelscited by 42
  33. 2017Neural Natural Language Inference Models Enhanced with External Knowledgecited by 40
  34. 2020Towards Robustifying NLI Models Against Lexical Dataset Biasescited by 34
  35. 2019What does BERT Learn from Multiple-Choice Reading Comprehension Datasets?cited by 30
  36. 2020A Unified Taylor Framework for Revisiting Attribution Methodscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 17; 2023: 22; 2024: 15; 2025: 13; 2026: 3.

78 papers.

Sort:

By area:

  1. 2023Explainability for Large Language Models: A Surveycited by 331
  2. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  3. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 96
  4. 2022Shortcut Learning of Large Language Models in Natural Language Understandingcited by 83
  5. 2021Identifying and Mitigating Spurious Correlations for Improving Robustness in NLP Modelscited by 76
  6. 2021"Will You Find These Shortcuts?" A Protocol for Evaluating the Faithfulness of Input Salience Methods for Text Classificationcited by 66
  7. 2022Detecting Shortcut Learning for Fair Medical AI using Shortcut Testingcited by 52
  8. 2023Won't Get Fooled Again: Answering Questions with False Premisescited by 47
  9. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 45
  10. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  11. 2021The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Failcited by 38
  12. 2024InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Modelscited by 36
38 more
  1. 2022PCL: Peer-Contrastive Learning with Diverse Augmentations for Unsupervised Sentence Embeddingscited by 29
  2. 2023What do neural networks learn in image classification? A frequency shortcut perspectivecited by 28
  3. 2025Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcutscited by 26
  4. 2021Introspective Distillation for Robust Question Answeringcited by 25
  5. 2022ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarizationcited by 25
  6. 2021Avoiding Inference Heuristics in Few-shot Prompt-based Finetuningcited by 23
  7. 2022Eye-gaze-guided Vision Transformer for Rectifying Shortcut Learningcited by 20
  8. 2023Distilling Cognitive Backdoor Patterns within an Imagecited by 20
  9. 2023A Survey on the Robustness of Computer Vision Models against Common Corruptionscited by 20
  10. 2024SLANG: New Concept Comprehension of Large Language Modelscited by 20
  11. 2021Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimationcited by 18
  12. 2024Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reductioncited by 18
  13. 2022Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typingcited by 16
  14. 2022Rectify ViT Shortcut Learning by Visual Saliencycited by 14
  15. 2024UFID: A Unified Framework for Input-level Backdoor Detection on Diffusion Modelscited by 14
  16. 2025Why and How LLMs Hallucinate: Connecting the Dots with Subsequence Associationscited by 14
  17. 2022Less Learn Shortcut: Analyzing and Mitigating Learning of Spurious Feature-Label Correlationcited by 13
  18. 2021Robustness Challenges in Model Distillation and Pruning for Natural Language Understandingcited by 12
  19. 2023A Comprehensive Survey of Sentence Representations: From the BERT Epoch to the ChatGPT Era and Beyondcited by 11
  20. 2024Co-occurrence is not Factual Association in Language Modelscited by 11
  21. 2022Feature-Level Debiased Natural Language Understandingcited by 10
  22. 2023Robust Natural Language Understanding with Residual Attention Debiasingcited by 10
  23. 2024Navigating the Shortcut Maze: A Comprehensive Analysis of Shortcut Learning in Text Classification by Language Modelscited by 10
  24. 2024Evaluating the Performance and Robustness of LLMs in Materials Science Q&A and Property Predictionscited by 9
  25. 2022XMD: An End-to-End Framework for Interactive Explanation-Based Debugging of NLP Modelscited by 8
  26. 2023Towards Robust Pruning: An Adaptive Knowledge-Retention Pruning Strategy for Language Modelscited by 7
  27. 2021Regularization for Long Named Entity Recognitioncited by 5
  28. 2022Debiasing Stance Detection Models with Counterfactual Reasoning and Adversarial Bias Learningcited by 5
  29. 2025Short-circuiting Shortcuts: Mechanistic Investigation of Shortcuts in Text Classificationcited by 5
  30. 2025Physics-informed Temporal Alignment for Auto-regressive PDE Foundation Modelscited by 5
  31. 2022AutoCAD: Automatically Generating Counterfactuals for Mitigating Shortcut Learningcited by 4
  32. 2023The Impacts of Unanswerable Questions on the Robustness of Machine Reading Comprehension Modelscited by 4
  33. 2023A Simple yet Effective Self-Debiasing Framework for Transformer Modelscited by 4
  34. 2023DFM-X: Augmentation by Leveraging Prior Knowledge of Shortcut Learningcited by 4
  35. 2024Learning Shortcuts: On the Misleading Promise of NLU in Language Modelscited by 4
  36. 2025Improving the OOD Performance of Closed-Source LLMs on NLI Through Strategic Data Selectioncited by 4
  37. 2022Pathologies of Pre-trained Language Models in Few-shot Fine-tuningcited by 3
  38. 2022Explaining Predictive Uncertainty by Looking Back at Model Explanationscited by 2

50 most cited of 78 All 78 (JSON)