mlsanityData to 7 Oct 2026

Certified Robustness to Adversarial Word Substitutions

arXiv 1909.00986 ↗First posted 3 Sep 2019cs.CL cs.LGcited by 164

Not on the map. Why?

Abstract (arXiv)

State-of-the-art NLP models can often be fooled by adversaries that apply seemingly innocuous label-preserving transformations (e.g., paraphrasing) to input text. The number of possible transformations scales exponentially with text length, so data augmentation cannot cover all transformations of an input. This paper considers one exponentially large family of label-preserving transformations, in which every word in the input can be replaced with a similar word. We train the first models that are provably robust to all word substitutions in this family. Our training procedure uses Interval Bound Propagation (IBP) to minimize an upper bound on the worst-case loss that any combination of word substitutions can induce. To evaluate models' robustness to these transformations, we measure accuracy on adversarially chosen word substitutions applied to test examples. Our IBP-trained models attain 75% adversarial accuracy on both sentiment analysis on IMDB and natural language inference on SNLI. In comparison, on IMDB, models trained normally and ones trained with data augmentation achieve adversarial accuracy of only 8% and 35%, respectively.

Key facts. Cited by 164 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 41; 2022: 42; 2023: 35; 2024: 19; 2025: 15; 2026: 12.

164 papers.

Sort:

By area:

  1. 2021A Survey of Data Augmentation Approaches for NLPcited by 408
  2. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 227
  3. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  4. 2021VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomenacited by 134
  5. 2021Entailment as Few-Shot Learnercited by 122
  6. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  7. 2022Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLPcited by 99
  8. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  9. 2021Towards Improving Adversarial Training of NLP Modelscited by 79
  10. 2021Bad Characters: Imperceptible NLP Attackscited by 69
  11. 2021Towards Robustness Against Natural Language Word Substitutionscited by 68
  12. 2023Black Box Adversarial Prompting for Foundation Modelscited by 62
38 more
  1. 2023Adversarial Demonstration Attacks on Large Language Modelscited by 58
  2. 2021Certified Robustness to Text Adversarial Attacks by Randomized [MASK]cited by 57
  3. 2021Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitutioncited by 55
  4. 2021On Guaranteed Optimal Robust Explanations for NLP Modelscited by 54
  5. 2021Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarkingcited by 53
  6. 2021How Should Pre-Trained Language Models Be Fine-Tuned Towards Adversarial Robustness?cited by 49
  7. 2023On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codexcited by 41
  8. 2023Text-CRS: A Generalized Certified Robustness Framework against Textual Adversarial Attackscited by 32
  9. 2021On the Sensitivity and Stability of Model Interpretations in NLPcited by 30
  10. 2021Defending Pre-trained Language Models from Adversarial Word Substitutions Without Performance Sacrificecited by 30
  11. 2023The Trickle-down Impact of Reward (In-)consistency on RLHFcited by 29
  12. 2021Grey-box Adversarial Attack And Defence For Sentiment Classificationcited by 28
  13. 2023Adversarial Illusions in Multi-Modal Embeddingscited by 28
  14. 2021Symbolic Brittleness in Sequence Models: on Systematic Generalization in Symbolic Mathematicscited by 27
  15. 2024Large Language Models and Causal Inference in Collaboration: A Surveycited by 26
  16. 2021How Robust are Model Rankings: A Leaderboard Customization Approach for Equitable Evaluationcited by 24
  17. 2022"That Is a Suspicious Reaction!": Interpreting Logits Variation to Detect NLP Adversarial Attackscited by 24
  18. 2021Achieving Model Robustness through Discrete Adversarial Trainingcited by 23
  19. 2022Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustnesscited by 23
  20. 2022Phrase-level Textual Adversarial Attack with Label Preservationcited by 23
  21. 2022Certified Robustness Against Natural Language Attacks by Causal Interventioncited by 23
  22. 2023Certified Robustness for Large Language Models with Self-Denoisingcited by 23
  23. 2021Self-Supervised Test-Time Learning for Reading Comprehensioncited by 22
  24. 2022A Survey of Adversarial Defences and Robustness in NLPcited by 22
  25. 2021Improving Zero-Shot Cross-Lingual Transfer Learning via Robust Trainingcited by 21
  26. 2021Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classificationcited by 20
  27. 2022Textual Manifold-based Defense Against Natural Language Adversarial Examplescited by 19
  28. 2021Certified Robustness to Programmable Transformations in LSTMscited by 18
  29. 2021The statistical advantage of automatic NLG metrics at the system levelcited by 18
  30. 2023How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attackscited by 18
  31. 2021On a Utilitarian Approach to Privacy Preserving Text Generationcited by 17
  32. 2021On the Robustness of Reading Comprehension Models to Entity Renamingcited by 17
  33. 2022Improving the Adversarial Robustness of NLP Models by Information Bottleneckcited by 17
  34. 2023RS-Del: Edit Distance Robustness Certificates for Sequence Classifiers via Randomized Deletioncited by 17
  35. 2021The King is Naked: on the Notion of Robustness for Natural Language Processingcited by 16
  36. 2022How Many Data Samples is an Additional Instruction Worth?cited by 16
  37. 2021Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigationcited by 15
  38. 2022On Robust Prefix-Tuning for Text Classificationcited by 15

50 most cited of 164 All 164 (JSON)