mlsanityData to 7 Oct 2026

Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP

arXiv 2210.10683 ↗First posted 19 Oct 2022cs.CL cs.CR cs.LGcited by 99

Not on the map. Why?

Cited by 53 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Textual adversarial samples play important roles in multiple subfields of NLP research, including security, evaluation, explainability, and data augmentation. However, most work mixes all these roles, obscuring the problem definitions and research goals of the security role that aims to reveal the practical concerns of NLP models. In this paper, we rethink the research paradigm of textual adversarial samples in security scenarios. We discuss the deficiencies in previous work and propose our suggestions that the research on the Security-oriented adversarial NLP (SoadNLP) should: (1) evaluate their methods on security tasks to demonstrate the real-world concerns; (2) consider real-world attackers' goals, instead of developing impractical methods. To this end, we first collect, process, and release a security datasets collection Advbench. Then, we reformalize the task and adjust the emphasis on different goals in SoadNLP. Next, we propose a simple method based on heuristic rules that can easily fulfill the actual adversarial goals to simulate real-world attack methods. We conduct experiments on both the attack and the defense sides on Advbench. Experimental results show that our method has higher practical value, indicating that the research paradigm in SoadNLP may start from our new benchmark. All the code and data of Advbench can be obtained at urlhttps://github.com/thunlp/Advbench.

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). Builds on 51 of its 96 references that are in the map. First posted on arXiv 19 Oct 2022.

Builds on

51 of 96 references.

References · 51

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,221
  3. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  4. 2017Automated Hate Speech Detection and the Problem of Offensive Languagecited by 724
  5. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  6. 2015Image-based Recommendations on Styles and Substitutescited by 702
  7. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  8. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  9. 2016Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filteringcited by 517
  10. 2020BERT-ATTACK: Adversarial Attack Against BERT Using BERTcited by 488
  11. 2018Generating Natural Language Adversarial Examplescited by 422
  12. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 341
38 more
  1. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  2. 2020Weight Poisoning Attacks on Pre-trained Modelscited by 318
  3. 2020BAE: BERT-based Adversarial Examples for Text Classificationcited by 316
  4. 2020Pretrained Transformers Improve Out-of-Distribution Robustnesscited by 293
  5. 2018Adversarial Example Generation with Syntactically Controlled Paraphrase Networkscited by 282
  6. 2019FreeLB: Enhanced Adversarial Training for Natural Language Understandingcited by 261
  7. 2017Deep Learning for Hate Speech Detection in Tweetscited by 180
  8. 2019Certified Robustness to Adversarial Word Substitutionscited by 164
  9. 2019Combating Adversarial Misspellings with Robust Word Recognitioncited by 162
  10. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfercited by 147
  11. 2018Deep Learning for Sentiment Analysis : A Surveycited by 139
  12. 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
  13. 2021Robustness Gym: Unifying the NLP Evaluation Landscapecited by 107
  14. 2016Crafting Adversarial Input Sequences for Recurrent Neural Networkscited by 102
  15. 2019Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagationcited by 87
  16. 2018On Adversarial Examples for Character-Level Neural Machine Translationcited by 85
  17. 2020OpenAttack: An Open-source Textual Adversarial Attack Toolkitcited by 84
  18. 2021Types of Out-of-Distribution Texts and How to Detect Themcited by 80
  19. 2020SAFER: A Structure-free Approach for Certified Robustness to Adversarial Word Substitutionscited by 72
  20. 2022A Unified Evaluation of Textual Backdoor Learning: Frameworks and Benchmarkscited by 70
  21. 2020Robust Encodings: A Framework for Combating Adversarial Typoscited by 58
  22. 2022Exploring the Universal Vulnerability of Prompt-based Learning Paradigmcited by 56
  23. 2017Towards Crafting Text Adversarial Samplescited by 55
  24. 2019Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classificationcited by 55
  25. 2020Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examplescited by 54
  26. 2019Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translationcited by 52
  27. 2018A Survey on Open Information Extractioncited by 51
  28. 2020Generating Natural Language Attacks in a Hard Label Black Box Settingcited by 50
  29. 2020CAT-Gen: Improving Robustness in NLP Models via Controlled Adversarial Text Generationcited by 43
  30. 2019Natural Language Adversarial Defense through Synonym Encodingcited by 39
  31. 2020Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuningcited by 39
  32. 2021Defending Pre-trained Language Models from Adversarial Word Substitutions Without Performance Sacrificecited by 30
  33. 2021Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Frameworkcited by 27
  34. 2021Achieving Model Robustness through Discrete Adversarial Trainingcited by 23
  35. 2020Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encodingcited by 22
  36. 2021Multi-granularity Textual Adversarial Attack with Behavior Cloningcited by 21
  37. 2021Textual Backdoor Attacks Can Be More Harmful via Two Simple Trickscited by 17
  38. 2017Satirical News Detection and Analysis using Attention Mechanism and Linguistic Featurescited by 14

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 14; 2024: 18; 2025: 38; 2026: 27.

99 papers.

Sort:

By area:

  1. 2024A StrongREJECT for Empty Jailbreakscited by 352
  2. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  3. 2024Efficient Adversarial Training in LLMs with Continuous Attackscited by 104
  4. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 96
  5. 2024Best-of-N Jailbreakingcited by 71
  6. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vectorcited by 63
  7. 2022A Close Look into the Calibration of Pre-trained Language Modelscited by 57
  8. 2025Representation Bending for Large Language Model Safetycited by 28
  9. 2023Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attackscited by 24
  10. 2023How do humans perceive adversarial text? A reality check on the validity and naturalness of word-based adversarial attackscited by 18
  11. 2025Understanding and Mitigating Overrefusal in LLMs from an Unveiling Perspective of Safety Decision Boundarycited by 17
  12. 2024HQA-Attack: Toward High Quality Black-Box Hard-Label Adversarial Attack on Textcited by 16
38 more
  1. 2024On the Intrinsic Self-Correction Capability of LLMs: Uncertainty and Latent Conceptcited by 16
  2. 2024Attack Atlas: A Practitioner's Perspective on Challenges and Pitfalls in Red Teaming GenAIcited by 14
  3. 2025Internal Value Alignment in Large Language Models through Controlled Value Vector Activationcited by 14
  4. 2025SDD: Self-Degraded Defense against Malicious Fine-tuningcited by 13
  5. 2023ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Modelscited by 12
  6. 2023Generating Valid and Natural Adversarial Examples with Large Language Modelscited by 12
  7. 2024Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxiacited by 12
  8. 2024Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Mergingcited by 11
  9. 2025Unsupervised Corpus Poisoning Attacks in Continuous Space for Dense Retrievalcited by 11
  10. 2025Evaluating Adversarial Robustness of Concept Representations in Sparse Autoencoderscited by 11
  11. 2025Evaluating Robustness of Large Audio Language Models to Audio Injection: An Empirical Studycited by 11
  12. 2025Mitigating the Safety Alignment Tax with Null-Space Constrained Policy Optimizationcited by 11
  13. 2023Guiding LLM to Fool Itself: Automatically Manipulating Machine Reading Comprehension Shortcut Triggerscited by 10
  14. 2025Enhancing Jailbreak Attacks on LLMs via Persona Promptscited by 10
  15. 2025GuidedBench: Measuring and Mitigating the Evaluation Discrepancies of In-the-wild LLM Jailbreak Methodscited by 9
  16. 2025LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challengescited by 9
  17. 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 9
  18. 2023Verifying the Robustness of Automatic Credibility Assessmentcited by 8
  19. 2024TAIA: Large Language Models are Out-of-Distribution Data Learnerscited by 8
  20. 2026GateMem: Benchmarking Memory Governance in Multi-Principal Shared-Memory Agentscited by 8
  21. 2025TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Lawcited by 7
  22. 2023CT-GAT: Cross-Task Generative Adversarial Attack based on Transferabilitycited by 6
  23. 2024Human-Interpretable Adversarial Prompt Attack on Large Language Models with Situational Contextcited by 6
  24. 2025EVOREFUSE: Evolutionary Prompt Optimization for Evaluation and Mitigation of LLM Over-Refusal to Pseudo-Malicious Instructionscited by 6
  25. 2025Anyone Can Jailbreak: Prompt-Based Attacks on LLMs and T2Iscited by 6
  26. 2026Paraphrasing Adversarial Attack on LLM-as-a-Reviewercited by 6
  27. 2024In-Context Learning Can Re-learn Forbidden Taskscited by 5
  28. 2025Extracting and Understanding the Superficial Knowledge in Alignmentcited by 5
  29. 2025Harmful Prompt Laundering: Jailbreaking LLMs with Abductive Styles and Symbolic Encodingcited by 5
  30. 2025SocialHarmBench: Revealing LLM Vulnerabilities to Socially Harmful Requestscited by 5
  31. 2025Backdoor Collapse: Eliminating Unknown Threats via Known Backdoor Aggregation in Language Modelscited by 5
  32. 2024FATS: A Prompt Injection Attack Utilizing Feign Security Agents with Deceptive Few-shots Learningcited by 4
  33. 2025SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoningcited by 4
  34. 2025TRIDENT: Enhancing Large Language Model Safety with Tri-Dimensional Diversified Red-Teaming Data Synthesiscited by 4
  35. 2025CAVGAN: Unifying Jailbreak and Defense of LLMs via Generative Adversarial Attacks on their Internal Representationscited by 4
  36. 2025SafeLLM: Unlearning Harmful Outputs from Large Language Models against Jailbreak Attackscited by 4
  37. 2025Investigating Safety Vulnerabilities of Large Audio-Language Models Under Speaker Emotional Variationscited by 4
  38. 2026AgenticRed: Evolving Agentic Systems for Red-Teamingcited by 4

50 most cited of 99 All 99 (JSON)