mlsanityData to 7 Oct 2026

Automatically Exposing Problems with Neural Dialog Models

arXiv 2109.06950 ↗First posted 14 Sep 2021cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Neural dialog models are known to suffer from problems such as generating unsafe and inconsistent responses. Even though these problems are crucial and prevalent, they are mostly manually identified by model designers through interactions. Recently, some research instructs crowdworkers to goad the bots into triggering such problems. However, humans leverage superficial clues such as hate speech, while leaving systematic problems undercover. In this paper, we propose two methods including reinforcement learning to automatically trigger a dialog model into generating problematic responses. We show the effect of our methods in exposing safety and contradiction issues with state-of-the-art dialog models.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 52 of its 60 references that are in the map. First posted on arXiv 14 Sep 2021.

Builds on

52 of 60 references.

References · 52

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  8. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  9. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  10. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  11. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  12. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
38 more
  1. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  2. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  3. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  4. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  5. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  6. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 754
  7. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  8. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 663
  9. 2020Recipes for building an open-domain chatbotcited by 593
  10. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  11. 2020Towards a Human-like Open-Domain Chatbotcited by 467
  12. 2019PAWS: Paraphrase Adversaries from Word Scramblingcited by 395
  13. 2020The Pushshift Reddit Datasetcited by 289
  14. 2016Deep Reinforcement Learning for Dialogue Generationcited by 286
  15. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 266
  16. 2016A Persona-Based Neural Conversation Modelcited by 221
  17. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  18. 2020Recipes for Safety in Open-domain Chatbotscited by 168
  19. 2017Learning to Generate Reviews and Discovering Sentimentcited by 147
  20. 2017Controlling Linguistic Style Aspects in Neural Language Generationcited by 147
  21. 2020SemEval-2020 Task 12: Multilingual Offensive Language Identification in Social Media (OffensEval 2020)cited by 138
  22. 2020Contextualized Perturbation for Textual Adversarial Attackcited by 136
  23. 2018Dialogue Natural Language Inferencecited by 135
  24. 2020Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skillscited by 130
  25. 2020Towards Controllable Biases in Language Generationcited by 112
  26. 2019Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Trainingcited by 110
  27. 2020Few-shot Natural Language Generation for Task-Oriented Dialogcited by 89
  28. 2020Multi-Dimensional Gender Bias Classificationcited by 80
  29. 2020I like fish, especially dolphins: Addressing Contradictions in Dialogue Modelingcited by 68
  30. 2019Evaluating Coherence in Dialogue Systems using Entailmentcited by 53
  31. 2020Universal Adversarial Attacks with Natural Triggers for Text Classificationcited by 50
  32. 2017A Deep Reinforcement Learning Chatbotcited by 36
  33. 2020Plug-and-Play Conversational Modelscited by 30
  34. 2021Attribute Alignment: Controlling Text Generation from Pre-trained Language Modelscited by 27
  35. 2019Negative Training for Neural Dialogue Response Generationcited by 24
  36. 2020Neural Generation Meets Real People: Towards Emotionally Engaging Mixed-Initiative Conversationscited by 24
  37. 2019Gunrock: A Social Bot for Complex and Engaging Long Conversationscited by 14
  38. 2018Detecting egregious responses in neural sequence-to-sequence modelscited by 6

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1; 2024: 2.

5 papers.

  1. 2022Red Teaming Language Models with Language Modelscited by 983
  2. 2023Fundamental Limitations of Alignment in Large Language Modelscited by 152
  3. 2024Tradeoffs Between Alignment and Helpfulness in Language Models with Steering Methodscited by 26
  4. 2024Legilimens: Practical and Unified Content Moderation for Large Language Model Servicescited by 13
  5. 2022Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generationcited by 8