mlsanityData to 7 Oct 2026

Recipes for Safety in Open-domain Chatbots

arXiv 2010.07079 ↗First posted 14 Oct 2020cs.CL cs.AIcited by 168

Not on the map. Why?

Cited by 29 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Models trained on large unlabeled corpora of human interactions will learn patterns and mimic behaviors therein, which include offensive or otherwise toxic behavior and unwanted biases. We investigate a variety of methods to mitigate these issues in the context of open-domain generative dialogue models. We introduce a new human-and-model-in-the-loop framework for both training safer models and for evaluating them, as well as a novel method to distill safety considerations inside generative models without the use of an external classifier at deployment time. We conduct experiments comparing these methods and find our new techniques are (i) safer than existing models as measured by automatic and human evaluations while (ii) maintaining usability metrics such as engagingness relative to the state of the art. We then discuss the limitations of this work by analyzing failure cases of our models.

Key facts. Cited by 168 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 41; 2023: 57; 2024: 31; 2025: 9; 2026: 7.

168 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  6. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  7. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  8. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  9. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  10. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 619
  11. 2023GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Ciphercited by 401
  12. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
38 more
  1. 2023CodeT5+: Open Code Large Language Models for Code Understanding and Generationcited by 327
  2. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
  3. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  4. 2023Pretraining Language Models with Human Preferencescited by 245
  5. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 242
  6. 2023Large Language Model Alignment: A Surveycited by 241
  7. 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
  8. 2021Internet-Augmented Dialogue Generationcited by 217
  9. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  10. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  11. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  12. 2021Challenges in Detoxifying Language Modelscited by 169
  13. 2023MART: Improving LLM Safety with Multi-round Automatic Red-Teamingcited by 161
  14. 2021DEMix Layers: Disentangling Domains for Modular Language Modelingcited by 130
  15. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 130
  16. 2022ProsocialDialog: A Prosocial Backbone for Conversational Agentscited by 109
  17. 2024On Prompt-Driven Safeguarding for Large Language Modelscited by 109
  18. 2021Detoxifying Language Models Risks Marginalizing Minority Voicescited by 106
  19. 2022Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completioncited by 102
  20. 2022Evaluating Human-Language Model Interactioncited by 90
  21. 2023Identifying and Mitigating the Security Risks of Generative AIcited by 90
  22. 2021Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoningcited by 85
  23. 2023Safety Assessment of Chinese Large Language Modelscited by 82
  24. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  25. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  26. 2022AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversationcited by 79
  27. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
  28. 2024WildVision: Evaluating Vision-Language Models in the Wild with Human Preferencescited by 76
  29. 2022COLD: A Benchmark for Chinese Offensive Language Detectioncited by 74
  30. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  31. 2023Attack Prompt Generation for Red Teaming and Defending Large Language Modelscited by 73
  32. 2024FlipAttack: Jailbreak LLMs via Flippingcited by 73
  33. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  34. 2022Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agentscited by 62
  35. 2023Prompted LLMs as Chatbot Modules for Long Open-domain Conversationcited by 61
  36. 2023Rewarding Chatbots for Real-World Engagement with Millions of Userscited by 58
  37. 2021Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contextscited by 56
  38. 2022Data Scaling Laws in NMT: The Effect of Noise and Architecturecited by 55

50 most cited of 168 All 168 (JSON)