mlsanityData to 7 Oct 2026

Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment

arXiv 1907.11932 ↗First posted 27 Jul 2019cs.CL cs.AI cs.LGcited by 663

Not on the map. Why?

Cited by 68 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Machine learning algorithms are often vulnerable to adversarial examples that have imperceptible alterations from the original counterparts but can fool the state-of-the-art models. It is helpful to evaluate or even improve the robustness of these models by exposing the maliciously crafted adversarial examples. In this paper, we present TextFooler, a simple but strong baseline to generate natural adversarial text. By applying it to two fundamental natural language tasks, text classification and textual entailment, we successfully attacked three target models, including the powerful pre-trained BERT, and the widely used convolutional and recurrent neural networks. We demonstrate the advantages of this framework in three ways: (1) effective---it outperforms state-of-the-art attacks in terms of success rate and perturbation rate, (2) utility-preserving---it preserves semantic content and grammaticality, and remains correctly classified by humans, and (3) efficient---it generates adversarial text with computational complexity linear to the text length. *The code, pre-trained target models, and test examples are available at https://github.com/jind11/TextFooler.

Key facts. Cited by 663 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 100; 2022: 117; 2023: 129; 2024: 147; 2025: 109; 2026: 61.

663 papers.

Sort:

By area:

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  2. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  3. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 457
  4. 2023Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrationscited by 401
  5. 2023Explainability for Large Language Models: A Surveycited by 331
  6. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  7. 2023Tool Learning with Foundation Modelscited by 302
  8. 2023On Evaluating Adversarial Robustness of Large Vision-Language Modelscited by 281
  9. 2022Prompting GPT-3 To Be Reliablecited by 258
  10. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 251
  11. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 233
  12. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 227
38 more
  1. 2024Tool Learning with Large Language Models: A Surveycited by 221
  2. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 204
  3. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 184
  4. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 175
  5. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  6. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 164
  7. 2024GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solverscited by 155
  8. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfercited by 147
  9. 2023Can Large Language Models Infer Causation from Correlation?cited by 142
  10. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 128
  11. 2021Split Computing and Early Exiting for Deep Learning Applications: Survey and Research Challengescited by 114
  12. 2021Towards Robustness of Text-to-SQL Models against Synonym Substitutioncited by 110
  13. 2021BERT: A Review of Applications in Natural Language Processing and Understandingcited by 109
  14. 2023Set-level Guidance Attack: Boosting Adversarial Transferability of Vision-Language Pre-training Modelscited by 106
  15. 2021On the Adversarial Robustness of Vision Transformerscited by 104
  16. 2022ReCode: Robustness Evaluation of Code Generation Modelscited by 100
  17. 2023DyVal: Dynamic Evaluation of Large Language Models for Reasoning Taskscited by 100
  18. 2022Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLPcited by 99
  19. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  20. 2024JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMscited by 87
  21. 2021Turn the Combination Lock: Learnable Textual Backdoor Attacks via Word Substitutioncited by 85
  22. 2022Shortcut Learning of Large Language Models in Natural Language Understandingcited by 83
  23. 2023Robust Multi-bit Natural Language Watermarking through Invariant Featurescited by 76
  24. 2023Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilitiescited by 76
  25. 2021Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Modelscited by 74
  26. 2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarkscited by 70
  27. 2023Watermarking Text Generated by Black-Box Language Modelscited by 68
  28. 2023VLATTACK: Multimodal Adversarial Attacks on Vision-Language Tasks via Pre-trained Modelscited by 66
  29. 2024Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbationscited by 64
  30. 2023Locally Differentially Private Document Generation Using Zero Shot Promptingcited by 63
  31. 2023How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Taskscited by 58
  32. 2023Adversarial Demonstration Attacks on Large Language Modelscited by 58
  33. 2021Certified Robustness to Text Adversarial Attacks by Randomized [MASK]cited by 57
  34. 2021Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNscited by 57
  35. 2021T-Miner: A Generative Approach to Defend Against Trojan Attacks on DNN-based Text Classificationcited by 56
  36. 2023A Bibliometric Review of Large Language Models Research from 2017 to 2023cited by 56
  37. 2021Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitutioncited by 55
  38. 2023Red Teaming Language Model Detectors with Language Modelscited by 55

50 most cited of 663 All 663 (JSON)