mlsanityData to 7 Oct 2026

Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution

arXiv 2108.12777 ↗First posted 29 Aug 2021cs.CLcited by 55

Not on the map. Why?

Abstract (arXiv)

Recent studies have shown that deep neural networks are vulnerable to intentionally crafted adversarial examples, and various methods have been proposed to defend against adversarial word-substitution attacks for neural NLP models. However, there is a lack of systematic study on comparing different defense approaches under the same attacking setting. In this paper, we seek to fill the gap of systematic studies through comprehensive researches on understanding the behavior of neural text classifiers trained by various defense methods under representative adversarial attacks. In addition, we propose an effective method to further improve the robustness of neural text classifiers against such attacks and achieved the highest accuracy on both clean and adversarial examples on AGNEWS and IMDB datasets by a significant margin.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 28 of its 41 references that are in the map. First posted on arXiv 29 Aug 2021.

Builds on

28 of 41 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  5. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  6. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  7. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  8. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  9. 2018Universal Sentence Encodercited by 485
  10. 2018Generating Natural Language Adversarial Examplescited by 422
  11. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 376
  12. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
16 more
  1. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  2. 2020Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examplescited by 275
  3. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  4. 2019Certified Robustness to Adversarial Word Substitutionscited by 164
  5. 2016Counter-fitting Word Vectors to Linguistic Constraintscited by 138
  6. 2020InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspectivecited by 92
  7. 2019Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagationcited by 87
  8. 2020Reevaluating Adversarial Examples in Natural Languagecited by 83
  9. 2020Robustness Verification for Transformerscited by 76
  10. 2020SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutionscited by 72
  11. 2021Towards Robustness Against Natural Language Word Substitutionscited by 68
  12. 2021Certified Robustness to Text Adversarial Attacks by Randomized [MASK]cited by 57
  13. 2018Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Datacited by 37
  14. 2020Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemblecited by 26
  15. 2020Searching for a Search Method: Benchmarking Search Algorithms for Generating NLP Adversarial Examplescited by 23
  16. 2020Voting based ensemble improves robustness of defensive modelscited by 7

Built on by

Cited by year

Citing papers per year: 2022: 16; 2023: 14; 2024: 16; 2025: 9.

55 papers.

Sort:

By area:

  1. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  2. 2022Bridging the Gap Between Indexing and Retrieval for Differentiable Search Index with Query Generationcited by 47
  3. 2022A Survey of Adversarial Defences and Robustness in NLPcited by 22
  4. 2022DABERT: Dual Attention Enhanced BERT for Semantic Matchingcited by 21
  5. 2022TextGrad: Advancing Robustness Evaluation in NLP by Gradient-Driven Optimizationcited by 21
  6. 2023Dual Path Modeling for Semantic Matching by Perceiving Subtle Conflictscited by 21
  7. 2022Textual Manifold-based Defense Against Natural Language Adversarial Examplescited by 19
  8. 2022Robust Lottery Tickets for Pre-trained Language Modelscited by 16
  9. 2022On Robust Prefix-Tuning for Text Classificationcited by 15
  10. 2022Text Adversarial Purification as Defense against Adversarial Attackscited by 15
  11. 2024NLP Verification: Towards a General Methodology for Certifying Robustnesscited by 14
  12. 2023Randomized Smoothing with Masked Inference for Adversarially Robust Text Classificationscited by 11
38 more
  1. 2024Spiking Convolutional Neural Networks for Text Classificationcited by 10
  2. 2022Impact of Adversarial Training on Robustness and Generalizability of Language Modelscited by 9
  3. 2023Model-tuning Via Prompts Makes NLP Models Adversarially Robustcited by 9
  4. 2024Comateformer: Combined Attention Transformer for Semantic Sentence Matchingcited by 9
  5. 2024Adversarial Text Purification: A Large Language Model Approach for Defensecited by 8
  6. 2023DSRM: Boost Textual Adversarial Training with Distribution Shift Risk Minimizationcited by 7
  7. 2024Fast Adversarial Training against Textual Adversarial Attackscited by 7
  8. 2023ANTONIO: Towards a Systematic Method of Generating NLP Benchmarks for Verificationcited by 6
  9. 2022Efficient Adversarial Training with Robust Early-Bird Ticketscited by 5
  10. 2023PerturbScore: Connecting Discrete and Continuous Perturbations in NLPcited by 5
  11. 2023IndoRobusta: Towards Robustness Against Diverse Code-Mixed Indonesian Local Languagescited by 5
  12. 2024ProTransformer: Robustify Transformers via Plug-and-Play Paradigmcited by 5
  13. 2022Identifying Adversarial Attacks on Text Classifierscited by 4
  14. 2022Improving robustness of language models from a geometry-aware perspectivecited by 4
  15. 2022Disentangled Text Representation Learning with Information-Theoretic Perspective for Adversarial Robustnesscited by 4
  16. 2023Robust Few-Shot Named Entity Recognition with Boundary Discrimination and Correlation Purificationcited by 4
  17. 2024Extreme Miscalibration and the Illusion of Adversarial Robustnesscited by 4
  18. 2024Evaluating and Safeguarding the Adversarial Robustness of Retrieval-Based In-Context Learningcited by 4
  19. 2022Don't sweat the small stuff, classify the rest: Sample Shielding to protect text classifiers against adversarial attackscited by 3
  20. 2023Enhancing Robustness of AI Offensive Code Generators via Data Augmentationcited by 3
  21. 2023RoAST: Robustifying Language Models via Adversarial Perturbation with Selective Trainingcited by 3
  22. 2024SemRoDe: Macro Adversarial Training to Learn Representations That are Robust to Word-Level Attackscited by 3
  23. 2022ADDMU: Detection of Far-Boundary Adversarial Examples with Data and Model Uncertainty Estimationcited by 2
  24. 2023Beyond Hard Samples: Robust and Effective Grammatical Error Correction with Cycle Self-Augmentingcited by 2
  25. 2024Explaining the role of Intrinsic Dimensionality in Adversarial Trainingcited by 2
  26. 2024MaskPure: Improving Defense Against Text Adversaries with Stochastic Purificationcited by 2
  27. 2025Tougher Text, Smarter Models: Raising the Bar for Adversarial Defence Benchmarkscited by 2
  28. 2025GenoArmory: A Unified Evaluation Framework for Adversarial Attacks on Genomic Foundation Modelscited by 2
  29. 2025Adversarial Defence without Adversarial Defence: Enhancing Language Model Robustness via Instance-level Principal Component Removalcited by 2
  30. 2025Guided Perturbation Sensitivity (GPS): Detecting Adversarial Text via Embedding Stability and Word Importancecited by 2
  31. 2022Generating Textual Adversaries with Minimal Perturbationcited by 1
  32. 2023Alert-ME: An Explainability-Driven Defense Against Adversarial Examples in Transformer-Based Text Classificationcited by 1
  33. 2023Fooling the Textual Fooler via Randomizing Latent Representationscited by 1
  34. 2024VertAttack: Taking advantage of Text Classifiers' horizontal visioncited by 1
  35. 2024DiffuseDef: Improved Robustness to Adversarial Attacks via Iterative Denoisingcited by 1
  36. 2025Bridging Robustness and Generalization Against Word Substitution Attacks in NLP via the Growth Bound Matrix Approachcited by 1
  37. 2025SALMAN: Stability Analysis of Language Models Through the Maps Between Graph-based Manifoldscited by 1
  38. 2025RobustMask: Certified Robustness against Adversarial Neural Ranking Attack via Randomized Maskingcited by 1

50 most cited of 55 All 55 (JSON)