mlsanityData to 7 Oct 2026

Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skills

arXiv 2004.08449 ↗First posted 17 Apr 2020cs.CLcited by 130

Not on the map. Why?

Abstract (arXiv)

Being engaging, knowledgeable, and empathetic are all desirable general qualities in a conversational agent. Previous work has introduced tasks and datasets that aim to help agents to learn those qualities in isolation and gauge how well they can express them. But rather than being specialized in one single quality, a good open-domain conversational agent should be able to seamlessly blend them all into one cohesive conversational flow. In this work, we investigate several ways to combine models trained towards isolated capabilities, ranging from simple model aggregation schemes that require minimal additional training, to various forms of multi-task training that encompass several skills at all training stages. We further propose a new dataset, BlendedSkillTalk, to analyze how these capabilities would mesh together in a natural conversation, and compare the performance of different architectures and training schemes. Our experiments show that multi-tasking over several tasks that focus on particular capabilities results in better blended conversation performance compared to models trained on a single skill, and that both unified or two-stage approaches perform well if they are constructed to avoid unwanted bias in skill selection or are fine-tuned on our new task.

Key facts. Cited by 130 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 28; 2022: 44; 2023: 25; 2024: 24; 2025: 4; 2026: 5.

130 papers.

Sort:

By area:

  1. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  2. 2023A Comprehensive Overview of Large Language Modelscited by 639
  3. 2022Knowledge Unlearning for Mitigating Privacy Risks in Language Modelscited by 424
  4. 2022OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalizationcited by 235
  5. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 197
  6. 2022SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualizationcited by 166
  7. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  8. 2021Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Surveycited by 130
  9. 2022ProsocialDialog: A Prosocial Backbone for Conversational Agentscited by 109
  10. 2022Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completioncited by 102
  11. 2022The AI Teacher Test: Measuring the Pedagogical Ability of Blender and GPT-3 in Educational Dialoguescited by 78
  12. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
38 more
  1. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  2. 2023PLACES: Prompting Language Models for Social Conversation Synthesiscited by 72
  3. 2021Few-Shot Bot: Prompt-Based Learning for Dialogue Systemscited by 64
  4. 2023Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMscited by 62
  5. 2021Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generationcited by 60
  6. 2021DynaEval: Unifying Turn and Dialogue Level Evaluationcited by 59
  7. 2024Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMscited by 59
  8. 2021Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contextscited by 56
  9. 2023Conversation Chronicles: Towards Diverse Temporal and Relational Dynamics in Multi-Session Conversationscited by 47
  10. 2021Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causescited by 43
  11. 2024To Each (Textual Sequence) Its Own: Improving Memorized-Data Unlearning in Large Language Modelscited by 42
  12. 2023Using In-Context Learning to Improve Dialogue Safetycited by 41
  13. 2021Conversations Are Not Flat: Modeling the Dynamic Information Flow across Dialogue Utterancescited by 40
  14. 2022The CRINGE Loss: Learning what language not to modelcited by 39
  15. 2022DIRECTOR: Generator-Classifiers For Supervised Language Modelingcited by 38
  16. 2024Digital Forgetting in Large Language Models: A Survey of Unlearning Methodscited by 38
  17. 2021PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generationcited by 37
  18. 2021SaFeRDialogues: Taking Feedback Gracefully after Conversational Safety Failurescited by 37
  19. 2024Selective Forgetting: Advancing Machine Unlearning Techniques and Evaluation in Language Modelscited by 36
  20. 2024Towards Robust and Parameter-Efficient Knowledge Unlearning for LLMscited by 36
  21. 2022Learning New Skills after Deployment: Improving open-domain internet-driven dialogue with human feedbackcited by 35
  22. 2022Meet Your Favorite Character: Open-domain Chatbot Mimicking Fictional Characters with only a Few Utterancescited by 32
  23. 2021Call for Customized Conversation: Customized Conversation Grounding Persona and Knowledgecited by 30
  24. 2023Ask an Expert: Leveraging Language Models to Improve Strategic Reasoning in Goal-Oriented Dialogue Modelscited by 30
  25. 2021Fusing task-oriented and open-domain dialogues in conversational agentscited by 28
  26. 2021Automatic Evaluation and Moderation of Open-domain Dialogue Systemscited by 28
  27. 2021Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue modelscited by 23
  28. 2021Exploring Prompt-based Few-shot Learning for Grounded Dialog Generationcited by 20
  29. 2022Empathic Conversations: A Multi-level Dataset of Contextualized Conversationscited by 20
  30. 2022When Life Gives You Lemons, Make Cherryade: Converting Feedback from Bad Responses into Good Labelscited by 20
  31. 2024Recent Trends in Personalized Dialogue Generation: A Review of Datasets, Methodologies, and Evaluationscited by 19
  32. 2022A Simple Contrastive Learning Objective for Alleviating Neural Text Degenerationcited by 18
  33. 2022DialogCC: An Automated Pipeline for Creating High-Quality Multi-Modal Dialogue Datasetcited by 17
  34. 2022MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussionscited by 17
  35. 2021Synthesizing Adversarial Negative Responses for Robust Response Ranking and Evaluationcited by 16
  36. 2022SalesBot: Transitioning from Chit-Chat to Task-Oriented Dialoguescited by 16
  37. 2023CHAMPAGNE: Learning Real-world Conversation from Large-Scale Web Videoscited by 14
  38. 2023Understanding Multi-Turn Toxic Behaviors in Open-Domain Chatbotscited by 14

50 most cited of 130 All 130 (JSON)