# Evaluating the Robustness of Neural Language Models to Input Perturbations

arXiv: https://arxiv.org/abs/2108.12237 · HTML: https://mlsanity.com/paper/2108.12237/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 37 references that are in the map.
- First posted on arXiv 27 Aug 2021.

## Builds on (most cited first, 25 of 25)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Universal Sentence Encoder](https://mlsanity.com/paper/1803.11175/) (2018)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies](https://mlsanity.com/paper/1611.01368/) (2016)
- [Synthetic and Natural Noise Both Break Neural Machine Translation](https://mlsanity.com/paper/1711.02173/) (2017)
- [Combating Adversarial Misspellings with Robust Word Recognition](https://mlsanity.com/paper/1905.11268/) (2019)
- [Deep Text Classification Can be Fooled](https://mlsanity.com/paper/1704.08006/) (2017)
- [Robustness of classifiers: from adversarial to random noise](https://mlsanity.com/paper/1608.08967/) (2016)
- [OpenAttack: An Open-source Textual Adversarial Attack Toolkit](https://mlsanity.com/paper/2009.09191/) (2020)
- [Perturbation Sensitivity Analysis to Detect Unintended Model Biases](https://mlsanity.com/paper/1910.04210/) (2019)
- [Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT](https://mlsanity.com/paper/2003.04985/) (2020)
- [Towards Robustness Against Natural Language Word Substitutions](https://mlsanity.com/paper/2107.13541/) (2021)
- [Robust Encodings: A Framework for Combating Adversarial Typos](https://mlsanity.com/paper/2005.01229/) (2020)
- [Evaluating Robustness to Input Perturbations for Neural Machine Translation](https://mlsanity.com/paper/2005.00580/) (2020)
- [How Robust Are Character-Based Word Embeddings in Tagging and MT Against Wrod Scramlbing or Randdm Nouse?](https://mlsanity.com/paper/1704.04441/) (2017)
- [Post-hoc explanation of black-box classifiers using confident itemsets](https://mlsanity.com/paper/2005.01992/) (2020)
- [Identify Susceptible Locations in Medical Records via Adversarial Attacks on Deep Predictive Models](https://mlsanity.com/paper/1802.04822/) (2018)
- [Explaining Black-box Models for Biomedical Text Classification](https://mlsanity.com/paper/2012.10928/) (2020)

## Built on by (most cited first, 30 of 87)

- [Prompt Injection attack against LLM-integrated Applications](https://mlsanity.com/paper/2306.05499/) (2023)
- [GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts](https://mlsanity.com/paper/2309.10253/) (2023)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [On Evaluating Adversarial Robustness of Large Vision-Language Models](https://mlsanity.com/paper/2305.16934/) (2023)
- [Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Text](https://mlsanity.com/paper/2202.06935/) (2022)
- [The Fallacy of AI Functionality](https://mlsanity.com/paper/2206.09511/) (2022)
- [Physics of Language Models: Part 1, Learning Hierarchical Language Structures](https://mlsanity.com/paper/2305.13673/) (2023)
- [Evaluation Gaps in Machine Learning Practice](https://mlsanity.com/paper/2205.05256/) (2022)
- [Changing Answer Order Can Decrease MMLU Accuracy](https://mlsanity.com/paper/2406.19470/) (2024)
- [In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPT](https://mlsanity.com/paper/2304.08979/) (2023)
- [PromptWizard: Task-Aware Prompt Optimization Framework](https://mlsanity.com/paper/2405.18369/) (2024)
- [Text-CRS: A Generalized Certified Robustness Framework against Textual Adversarial Attacks](https://mlsanity.com/paper/2307.16630/) (2023)
- [Large Language Models for UAVs: Current State and Pathways to the Future](https://mlsanity.com/paper/2405.01745/) (2024)
- [Assessing Hidden Risks of LLMs: An Empirical Study on Robustness, Consistency, and Credibility](https://mlsanity.com/paper/2305.10235/) (2023)
- [Revisit Input Perturbation Problems for LLMs: A Unified Robustness Evaluation Framework for Noisy Slot Filling Task](https://mlsanity.com/paper/2310.06504/) (2023)
- [Can LLMs Recognize Toxicity? A Structured Investigation Framework and Toxicity Metric](https://mlsanity.com/paper/2402.06900/) (2024)
- [Assessing Dialect Fairness and Robustness of Large Language Models in Reasoning Tasks](https://mlsanity.com/paper/2410.11005/) (2024)
- [Brittlebench: Quantifying LLM robustness via prompt sensitivity](https://mlsanity.com/paper/2603.13285/) (2026)
- [Rank List Sensitivity of Recommender Systems to Interaction Perturbations](https://mlsanity.com/paper/2201.12686/) (2022)
- [Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval](https://mlsanity.com/paper/2212.10526/) (2022)
- [RobustFlow: Towards Robust Agentic Workflow Generation](https://mlsanity.com/paper/2509.21834/) (2025)
- [Interpreting the Robustness of Neural NLP Models to Textual Perturbations](https://mlsanity.com/paper/2110.07159/) (2021)
- [Not Just Pretty Pictures: Toward Interventional Data Augmentation Using Text-to-Image Generators](https://mlsanity.com/paper/2212.11237/) (2022)
- [Enhancing LLM Robustness to Perturbed Instructions: An Empirical Study](https://mlsanity.com/paper/2504.02733/) (2025)
- [Influence of External Information on Large Language Models Mirrors Social Cognitive Patterns](https://mlsanity.com/paper/2305.04812/) (2023)
- [NLP Verification: Towards a General Methodology for Certifying Robustness](https://mlsanity.com/paper/2403.10144/) (2024)
- [Invariant Language Modeling](https://mlsanity.com/paper/2110.08413/) (2021)
- [Towards Truly Zero-shot Compositional Visual Reasoning with LLMs as Programmers](https://mlsanity.com/paper/2401.01974/) (2024)
- [Prompt Perturbations Reveal Human-Like Biases in Large Language Model Survey Responses](https://mlsanity.com/paper/2507.07188/) (2025)
- [Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions](https://mlsanity.com/paper/2201.00768/) (2022)

## Abstract (arXiv)

High-performance neural language models have obtained state-of-the-art results on a wide range of Natural Language Processing (NLP) tasks. However, results for common benchmark datasets often do not reflect model reliability and robustness when applied to noisy, real-world data. In this study, we design and implement various types of character-level and word-level perturbation methods to simulate realistic scenarios in which input texts may be slightly noisy or different from the data distribution on which NLP systems were trained. Conducting comprehensive experiments on different NLP tasks, we investigate the ability of high-performance language models such as BERT, XLNet, RoBERTa, and ELMo in handling different types of input perturbations. The results suggest that language models are sensitive to input perturbations and their performance can decrease even when small changes are introduced. We highlight that models need to be further improved and that current benchmarks are not reflecting model robustness well. We argue that evaluations on perturbed inputs should routinely complement widely-used benchmarks in order to yield a more realistic understanding of NLP systems robustness.