mlsanityData to 14 Oct 2026

High-Quality Diversification for Task-Oriented Dialogue Systems

arXiv 2106.00891 ↗First posted 2 Jun 2021cs.CL cs.HCcited by 3

Not on the map. Why?

Abstract (arXiv)

Many task-oriented dialogue systems use deep reinforcement learning (DRL) to learn policies that respond to the user appropriately and complete the tasks successfully. Training DRL agents with diverse dialogue trajectories prepare them well for rare user requests and unseen situations. One effective diversification method is to let the agent interact with a diverse set of learned user models. However, trajectories created by these artificial user models may contain generation errors, which can quickly propagate into the agent's policy. It is thus important to control the quality of the diversification and resist the noise. In this paper, we propose a novel dialogue diversification method for task-oriented dialogue systems trained in simulators. Our method, Intermittent Short Extension Ensemble (I-SEE), constrains the intensity to interact with an ensemble of diverse user models and effectively controls the quality of the diversification. Evaluations on the Multiwoz dataset show that I-SEE successfully boosts the performance of several state-of-the-art DRL dialogue agents.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 33 of its 43 references that are in the map. First posted on arXiv 2 Jun 2021.

Builds on

33 of 43 references.

References · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  3. 2017Hindsight Experience Replaycited by 1,136
  4. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,077
  5. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  6. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  7. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  8. 2018Behavioral Cloning from Observationcited by 492
  9. 2020A Simple Language Model for Task-Oriented Dialoguecited by 332
  10. 2015A Neural Conversational Modelcited by 196
  11. 2019Asking Clarifying Questions in Open-Domain Information-Seeking Conversationscited by 139
  12. 2020Open-Retrieval Conversational Question Answeringcited by 99
21 more
  1. 2020Few-Shot Generative Conversational Query Rewritingcited by 85
  2. 2018Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systemscited by 70
  3. 2020Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Trackingcited by 65
  4. 2016A User Simulator for Task-Completion Dialoguescited by 57
  5. 2017End-to-End Task-Completion Neural Dialogue Systemscited by 52
  6. 2020ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systemscited by 52
  7. 2019How to Build User Simulators to Train RL-based Dialog Systemscited by 46
  8. 2017Ask the Right Questions: Active Question Reformulation with Reinforcement Learningcited by 44
  9. 2018Learning To Simulatecited by 44
  10. 2019Look before you Hop: Conversational Question Answering over Knowledge Graphs Using Judicious Context Expansioncited by 42
  11. 2019Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialogcited by 37
  12. 2016Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Accesscited by 36
  13. 2020Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systemscited by 35
  14. 2018Generating More Interesting Responses in Neural Conversation Models with Distributional Constraintscited by 33
  15. 2017BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systemscited by 30
  16. 2020Multi-Agent Task-Oriented Dialog Policy Learning with Role-Aware Reward Decompositioncited by 23
  17. 2019Automatically Learning Data Augmentation Policies for Dialogue Taskscited by 20
  18. 2018Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learningcited by 11
  19. 2018Goal-oriented Dialogue Policy Learning from Failurescited by 8
  20. 2021Controlling the Risk of Conversational Search via Reinforcement Learningcited by 8
  21. 2019Corpus-Level End-to-End Exploration for Interactive Systemscited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1.

3 papers.

  1. 2023EmoUS: Simulating User Emotions in Task-Oriented Dialoguescited by 9
  2. 2024Knowledge-Enhanced Conversational Recommendation via Transformer-based Sequential Modellingcited by 5
  3. 2023An Asynchronous Updating Reinforcement Learning Framework for Task-oriented Dialog Systemcited by 0