# Reevaluating Adversarial Examples in Natural Language

arXiv: https://arxiv.org/abs/2004.14174 · HTML: https://mlsanity.com/paper/2004.14174/

## Key facts

- Cited by 85 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2020.

## Built on by (most cited first, 30 of 85)

- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [On Evaluating Adversarial Robustness of Large Vision-Language Models](https://mlsanity.com/paper/2305.16934/) (2023)
- [Gradient-based Adversarial Attacks against Text Transformers](https://mlsanity.com/paper/2104.13733/) (2021)
- [Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models](https://mlsanity.com/paper/2111.02840/) (2021)
- [Measure and Improve Robustness in NLP Models: A Survey](https://mlsanity.com/paper/2112.08313/) (2021)
- [Robust Multi-bit Natural Language Watermarking through Invariant Features](https://mlsanity.com/paper/2305.01904/) (2023)
- [Towards Improving Adversarial Training of NLP Models](https://mlsanity.com/paper/2109.00544/) (2021)
- [A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarks](https://mlsanity.com/paper/2206.08514/) (2022)
- [Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution](https://mlsanity.com/paper/2108.12777/) (2021)
- [The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail](https://mlsanity.com/paper/2110.08300/) (2021)
- [Detection of Word Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimation](https://mlsanity.com/paper/2203.01677/) (2022)
- [Grey-box Adversarial Attack And Defence For Sentiment Classification](https://mlsanity.com/paper/2103.11576/) (2021)
- [Robust Conversational Agents against Imperceptible Toxicity Triggers](https://mlsanity.com/paper/2205.02392/) (2022)
- [Phrase-level Textual Adversarial Attack with Label Preservation](https://mlsanity.com/paper/2205.10710/) (2022)
- [MTTM: Metamorphic Testing for Textual Content Moderation Software](https://mlsanity.com/paper/2302.05706/) (2023)
- [PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMs](https://mlsanity.com/paper/2409.14866/) (2024)
- [How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attacks](https://mlsanity.com/paper/2305.15587/) (2023)
- [The King is Naked: on the Notion of Robustness for Natural Language Processing](https://mlsanity.com/paper/2112.07605/) (2021)
- [Text Adversarial Purification as Defense against Adversarial Attacks](https://mlsanity.com/paper/2203.14207/) (2022)
- [Adversarial Decoding: Generating Readable Documents for Adversarial Objectives](https://mlsanity.com/paper/2410.02163/) (2024)
- [On The Empirical Effectiveness of Unrealistic Adversarial Hardening Against Realistic Adversarial Attacks](https://mlsanity.com/paper/2202.03277/) (2022)
- [Towards Explainable Evaluation Metrics for Natural Language Generation](https://mlsanity.com/paper/2203.11131/) (2022)
- [AEON: A Method for Automatic Evaluation of NLP Test Cases](https://mlsanity.com/paper/2205.06439/) (2022)
- [Towards Building a Robust Toxicity Predictor](https://mlsanity.com/paper/2404.08690/) (2024)
- [Less Learn Shortcut: Analyzing and Mitigating Learning of Spurious Feature-Label Correlation](https://mlsanity.com/paper/2205.12593/) (2022)
- [Token-Modification Adversarial Attacks for Natural Language Processing: A Survey](https://mlsanity.com/paper/2103.00676/) (2021)
- [TransFool: An Adversarial Attack against Neural Machine Translation Models](https://mlsanity.com/paper/2302.00944/) (2023)
- [Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Models](https://mlsanity.com/paper/2503.01742/) (2025)
- [BERT is Robust! A Case Against Synonym-Based Adversarial Examples in Text Classification](https://mlsanity.com/paper/2109.07403/) (2021)

## Abstract (arXiv)

State-of-the-art attacks on NLP models lack a shared definition of a what constitutes a successful attack. We distill ideas from past work into a unified framework: a successful natural language adversarial example is a perturbation that fools the model and follows some linguistic constraints. We then analyze the outputs of two state-of-the-art synonym substitution attacks. We find that their perturbations often do not preserve semantics, and 38% introduce grammatical errors. Human surveys reveal that to successfully preserve semantics, we need to significantly increase the minimum cosine similarities between the embeddings of swapped words and between the sentence encodings of original and perturbed sentences.With constraints adjusted to better preserve semantics and grammaticality, the attack success rate drops by over 70 percentage points.