mlsanityData to 14 Oct 2026

Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack

arXiv 1908.06083 ↗First posted 17 Aug 2019cs.CLcited by 176

Not on the map. Why?

Cited by 41 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The detection of offensive language in the context of a dialogue has become an increasingly important application of natural language processing. The detection of trolls in public forums (Gal'an-Garc'ia et al., 2016), and the deployment of chatbots in the public domain (Wolf et al., 2017) are two examples that show the necessity of guarding against adversarially offensive behavior on the part of humans. In this work, we develop a training scheme for a model to become robust to such human attacks by an iterative build it, break it, fix it strategy with humans and models in the loop. In detailed experiments we show this approach is considerably more robust than previous systems. Further, we show that offensive language used within a conversation critically depends on the dialogue context, and cannot be viewed as a single sentence offensive detection task as in most previous work. Our newly collected tasks and methods will be made open source and publicly available.

Key facts. Cited by 176 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 31; 2022: 40; 2023: 43; 2024: 38; 2025: 18; 2026: 6.

176 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  3. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  5. 2023PaLM 2 Technical Reportcited by 1,162
  6. 2022Red Teaming Language Models with Language Modelscited by 1,026
  7. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  8. 2022Improving alignment of dialogue agents via targeted human judgementscited by 531
  9. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  10. 2021Dynabench: Rethinking Benchmarking in NLPcited by 459
  11. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  12. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
38 more
  1. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 291
  2. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 264
  3. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  4. 2024ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMscited by 216
  5. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 202
  6. 2023SafetyBench: Evaluating the Safety of Large Language Modelscited by 199
  7. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  8. 2023Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attackscited by 194
  9. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 163
  10. 2024AutoDefense: Multi-Agent LLM Defense against Jailbreak Attackscited by 145
  11. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 121
  12. 2022ProsocialDialog: A Prosocial Backbone for Conversational Agentscited by 109
  13. 2021Challenges in Automated Debiasing for Toxic Language Detectioncited by 96
  14. 2021Human-in-the-Loop for Data Collection: a Multi-Target Counter Narrative Dataset to Fight Online Hate Speechcited by 82
  15. 2021On Measures of Biases and Harms in NLPcited by 81
  16. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 78
  17. 2024Safeguarding Large Language Models: A Surveycited by 78
  18. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 77
  19. 2022COLD: A Benchmark for Chinese Offensive Language Detectioncited by 76
  20. 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completioncited by 76
  21. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  22. 2023Eight Things to Know about Large Language Modelscited by 73
  23. 2023PLACES: Prompting Language Models for Social Conversation Synthesiscited by 72
  24. 2023A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospectscited by 66
  25. 2021Few-Shot Bot: Prompt-Based Learning for Dialogue Systemscited by 64
  26. 2022Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agentscited by 62
  27. 2024Introducing v0.5 of the AI Safety Benchmark from MLCommonscited by 62
  28. 2022Adversarial Training for High-Stakes Reliabilitycited by 60
  29. 2021Human-Adversarial Visual Question Answeringcited by 57
  30. 2022AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Modelcited by 57
  31. 2021Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contextscited by 56
  32. 2023The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Valuescited by 54
  33. 2023AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applicationscited by 54
  34. 2024Learning diverse attacks on large language models for robust red-teaming and safety tuningcited by 54
  35. 2022Perturbation Augmentation for Fairer NLPcited by 52
  36. 2023Unveiling the Implicit Toxicity in Large Language Modelscited by 51
  37. 2024CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreferencecited by 51
  38. 2021Shades of BLEU, Flavours of Success: The Case of MultiWOZcited by 48

50 most cited of 176 All 176 (JSON)