mlsanityData to 14 Oct 2026

Universal Adversarial Triggers for Attacking and Analyzing NLP

arXiv 1908.07125 ↗First posted 20 Aug 2019cs.CL cs.LGcited by 722

Not on the map. Why?

Cited by 186 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Adversarial examples highlight model vulnerabilities and are useful for evaluation and interpretation. We define universal adversarial triggers: input-agnostic sequences of tokens that trigger a model to produce a specific prediction when concatenated to any input from a dataset. We propose a gradient-guided search over tokens which finds short trigger sequences (e.g., one word for classification and four words for language modeling) that successfully trigger the target prediction. For example, triggers cause SNLI entailment accuracy to drop from 89.94% to 0.55%, 72% of "why" questions in SQuAD to be answered "to kill american people", and the GPT-2 language model to spew racist output even when conditioned on non-racial contexts. Furthermore, although the triggers are optimized using white-box access to a specific model, they transfer to other models for all tasks we consider. Finally, since triggers are input-agnostic, they provide an analysis of global model behavior. For instance, they confirm that SNLI models exploit dataset biases and help to diagnose heuristics learned by reading comprehension models.

Key facts. Cited by 722 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 93; 2022: 118; 2023: 135; 2024: 153; 2025: 133; 2026: 90.

722 papers.

Sort:

By area:

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  2. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  3. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  4. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  5. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  6. 2022Red Teaming Language Models with Language Modelscited by 1,026
  7. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 739
  8. 2021Deduplicating Training Data Makes Language Models Bettercited by 714
  9. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 651
  10. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  11. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 509
  12. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
38 more
  1. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  2. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
  3. 2021Dynabench: Rethinking Benchmarking in NLPcited by 459
  4. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
  5. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 388
  6. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  7. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  8. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 366
  9. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
  10. 2023Visual Adversarial Examples Jailbreak Aligned Large Language Modelscited by 328
  11. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  12. 2023Are aligned neural networks adversarially aligned?cited by 315
  13. 2023Low-Resource Languages Jailbreak GPT-4cited by 309
  14. 2023Tool Learning with Foundation Modelscited by 302
  15. 2021Factual Probing Is [MASK]: Learning vs. Learning to Recallcited by 291
  16. 2023Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Modelscited by 273
  17. 2023Poisoning Language Models During Instruction Tuningcited by 269
  18. 2022Prompting GPT-3 To Be Reliablecited by 260
  19. 2023Large Language Model Alignment: A Surveycited by 243
  20. 2021Gradient-based Adversarial Attacks against Text Transformerscited by 234
  21. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 226
  22. 2021Get Your Vitamin C! Robust Fact Verification with Contrastive Evidencecited by 225
  23. 2024Tool Learning with Large Language Models: A Surveycited by 223
  24. 2023Data-centric Artificial Intelligence: A Surveycited by 203
  25. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  26. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 202
  27. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 194
  28. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
  29. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 193
  30. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 191
  31. 2022PointCLIP V2: Prompting CLIP and GPT for Powerful 3D Open-world Learningcited by 187
  32. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 185
  33. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 180
  34. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  35. 2024COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllabilitycited by 175
  36. 2021Challenges in Detoxifying Language Modelscited by 169
  37. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 159
  38. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 156

50 most cited of 722 All 722 (JSON)