mlsanityData to 7 Oct 2026

Measure and Improve Robustness in NLP Models: A Survey

arXiv 2112.08313 ↗First posted 15 Dec 2021cs.CL cs.LGcited by 107

Not on the map. Why?

Abstract (arXiv)

As NLP models achieved state-of-the-art performances over benchmarks and gained wide applications, it has been increasingly important to ensure the safe deployment of these models in the real world, e.g., making sure the models are robust against unseen or challenging scenarios. Despite robustness being an increasingly studied topic, it has been separately explored in applications like vision and NLP, with various definitions, evaluation and mitigation strategies in multiple lines of research. In this paper, we aim to provide a unifying survey of how to define, measure and improve robustness in NLP. We first connect multiple definitions of robustness, then unify various lines of work on identifying robustness failures and evaluating models' robustness. Correspondingly, we present mitigation strategies that are data-driven, model-driven, and inductive-prior-based, with a more systematic view of how to effectively improve robustness in NLP models. Finally, we conclude by outlining open challenges and future directions to motivate further research in this area.

Key facts. Cited by 107 papers in the map (data through 7 Oct 2026). Builds on 147 of its 189 references that are in the map. First posted on arXiv 15 Dec 2021.

Builds on

147 of 189 references.

References · 147

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2017On Calibration of Modern Neural Networkscited by 4,860
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  7. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  8. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  9. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  10. 2019Invariant Risk Minimizationcited by 1,832
  11. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  12. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
38 more
  1. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  2. 2019Natural Adversarial Examplescited by 1,401
  3. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  4. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  5. 2020In Search of Lost Domain Generalizationcited by 1,042
  6. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,010
  7. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  8. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  9. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 897
  10. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  11. 2018Robustness May Be at Odds with Accuracycited by 873
  12. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  13. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  14. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  15. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  16. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  17. 2013Domain Generalization via Invariant Feature Representationcited by 622
  18. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  19. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
  20. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  21. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  22. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 452
  23. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  24. 2018Gender Bias in Coreference Resolutioncited by 438
  25. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  26. 2018Generating Natural Language Adversarial Examplescited by 422
  27. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  28. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 395
  29. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  30. 2018A Survey of Unsupervised Deep Domain Adaptationcited by 361
  31. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  32. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
  33. 2018SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inferencecited by 333
  34. 2017Unified Deep Supervised Domain Adaptation and Generalizationcited by 322
  35. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  36. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  37. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  38. 2020An Investigation of Why Overparameterization Exacerbates Spurious Correlationscited by 310

50 most cited of 147 All 147 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 14; 2023: 46; 2024: 21; 2025: 19; 2026: 7.

107 papers.

Sort:

By area:

  1. 2023State of What Art? A Call for Multi-Prompt LLM Evaluationcited by 299
  2. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 96
  3. 2023Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debatecited by 92
  4. 2024Safeguarding Large Language Models: A Surveycited by 73
  5. 2022GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspectivecited by 72
  6. 2023Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmarkcited by 62
  7. 2023RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbationscited by 41
  8. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  9. 2026AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocationscited by 33
  10. 2024Examining the robustness of LLM evaluation to the distributional assumptions of benchmarkscited by 31
  11. 2023The Trickle-down Impact of Reward (In-)consistency on RLHFcited by 29
  12. 2023LLMs as Workers in Human-Computational Algorithms? Replicating Crowdsourcing Pipelines with LLMscited by 27
38 more
  1. 2022Robustness Analysis of Video-Language Models Against Visual and Language Perturbationscited by 23
  2. 2022CORE: A Retrieve-then-Edit Framework for Counterfactual Data Generationcited by 23
  3. 2022A Survey of Methods for Addressing Class Imbalance in Deep-Learning Based Natural Language Processingcited by 20
  4. 2024Speech Robust Bench: A Robustness Benchmark For Speech Recognitioncited by 20
  5. 2024SemEval-2024 Task 2: Safe Biomedical Natural Language Inference for Clinical Trialscited by 20
  6. 2024Grounding and Evaluation for Large Language Models: Practical Challenges and Lessons Learned (Survey)cited by 20
  7. 2022Not Just Pretty Pictures: Toward Interventional Data Augmentation Using Text-to-Image Generatorscited by 17
  8. 2024Unveiling LLM Evaluation Focused on Metrics: Challenges and Solutionscited by 17
  9. 2025Enhancing LLM Robustness to Perturbed Instructions: An Empirical Studycited by 17
  10. 2023Setting the Trap: Capturing and Defeating Backdoors in Pretrained Language Models through Honeypotscited by 16
  11. 2024Unsupervised Concept Drift Detection from Deep Learning Representations in Real-timecited by 16
  12. 2025RAS: Retrieval-And-Structuring for Knowledge-Intensive LLM Generationcited by 16
  13. 2024Robust Neural Information Retrieval: An Adversarial and Out-of-distribution Perspectivecited by 15
  14. 2023SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Modelscited by 14
  15. 2024NLP Verification: Towards a General Methodology for Certifying Robustnesscited by 14
  16. 2024Beyond Interpretability: The Gains of Feature Monosemanticity on Model Robustnesscited by 13
  17. 2023Through the Lens of Core Competency: Survey on Evaluation of Large Language Modelscited by 12
  18. 2022Robustness of Demonstration-based Learning Under Limited Data Scenariocited by 11
  19. 2022TAPE: Assessing Few-shot Russian Language Understandingcited by 11
  20. 2023Measuring the Robustness of NLP Models to Domain Shiftscited by 11
  21. 2023The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Modelscited by 11
  22. 2022SynSciPass: detecting appropriate uses of scientific text generationcited by 10
  23. 2025FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypescited by 9
  24. 2023Robust Question Answering against Distribution Shifts with Test-Time Adaptation: An Empirical Studycited by 8
  25. 2023Don't Retrain, Just Rewrite: Countering Adversarial Perturbations by Rewriting Textcited by 8
  26. 2023Generalization Analogies: A Testbed for Generalizing AI Oversight to Hard-To-Measure Domainscited by 8
  27. 2022Towards Generalizable and Robust Text-to-SQL Parsingcited by 7
  28. 2023ANTONIO: Towards a Systematic Method of Generating NLP Benchmarks for Verificationcited by 6
  29. 2023Multi-Source Test-Time Adaptation as Dueling Bandits for Extractive Question Answeringcited by 6
  30. 2023LEAP: Efficient and Automated Test Method for NLP Softwarecited by 6
  31. 2023RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbationcited by 6
  32. 2023Evaluating Concurrent Robustness of Language Models Across Diverse Challenge Setscited by 6
  33. 2023Generative Data Augmentation using LLMs improves Distributional Robustness in Question Answeringcited by 5
  34. 2023Robustness Tests for Automatic Machine Translation Metrics with Adversarial Attackscited by 5
  35. 2023Whispers of Doubt Amidst Echoes of Triumph in NLP Robustnesscited by 5
  36. 2024LLM-Based Robust Product Classification in Commerce and Compliancecited by 5
  37. 2025None of the Others: a General Technique to Distinguish Reasoning from Memorization in Multiple-Choice LLM Evaluation Benchmarkscited by 5
  38. 2025Evaluating and Improving Robustness in Large Language Models: A Survey and Future Directionscited by 5

50 most cited of 107 All 107 (JSON)