mlsanityData to 14 Oct 2026

Transferable Dialogue Systems and User Simulators

arXiv 2107.11904 ↗First posted 25 Jul 2021cs.CL cs.AIcited by 36

Not on the map. Why?

Abstract (arXiv)

One of the difficulties in training dialogue systems is the lack of training data. We explore the possibility of creating dialogue data through the interaction between a dialogue system and a user simulator. Our goal is to develop a modelling framework that can incorporate new dialogue scenarios through self-play between the two agents. In this framework, we first pre-train the two agents on a collection of source domain dialogues, which equips the agents to converse with each other via natural language. With further fine-tuning on a small amount of target domain data, the agents continue to interact with the aim of improving their behaviors using reinforcement learning with structured reward functions. In experiments on the MultiWOZ dataset, two practical transfer learning problems are investigated: 1) domain adaptation and 2) single-to-multiple domain transfer. We demonstrate that the proposed framework is highly effective in bootstrapping the performance of the two agents in transfer learning. We also show that our method leads to improvements in dialogue system performance on complete datasets.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Builds on 30 of its 40 references that are in the map. First posted on arXiv 25 Jul 2021.

Builds on

30 of 40 references.

References · 30

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  4. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 766
  5. 2020A Simple Language Model for Task-Oriented Dialoguecited by 332
  6. 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
  7. 2015Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Modelscited by 207
  8. 2020MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systemscited by 103
  9. 2019Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Contextcited by 99
  10. 2020SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teachingcited by 71
  11. 2016A User Simulator for Task-Completion Dialoguescited by 57
  12. 2019Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Modelscited by 53
18 more
  1. 2017End-to-End Task-Completion Neural Dialogue Systemscited by 52
  2. 2020Paraphrase Augmented Task-Oriented Dialog Generationcited by 43
  3. 2018Neural User Simulation for Corpus-based Policy Optimisation for Spoken Dialogue Systemscited by 42
  4. 2019Structured Fusion Networks for Dialogcited by 40
  5. 2016A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systemscited by 39
  6. 2016Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Accesscited by 36
  7. 2018User Modeling for Task Oriented Dialoguescited by 36
  8. 2019ConvLab: Multi-Domain End-to-End Dialog System Platformcited by 29
  9. 2017Iterative Policy Learning in End-to-End Trainable Task-Oriented Neural Dialog Modelscited by 26
  10. 2020Multi-Domain Dialogue Acts and Response Co-Generationcited by 26
  11. 2019Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Modelscited by 25
  12. 2020Multi-Agent Task-Oriented Dialog Policy Learning with Role-Aware Reward Decompositioncited by 23
  13. 2017End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learningcited by 19
  14. 2017Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Managementcited by 18
  15. 2017An End-to-End Trainable Neural Network Model with Belief Tracking for Task-Oriented Dialogcited by 17
  16. 2019Collaborative Multi-Agent Dialogue Model Training Via Reinforcement Learningcited by 9
  17. 2019Teacher-Student Framework Enhanced Multi-domain Dialogue Generationcited by 8
  18. 2019Retrospective and Prospective Mixture-of-Generators for Task-oriented Dialogue Response Generationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 15; 2023: 8; 2024: 7; 2025: 3; 2026: 2.

36 papers.

Sort:

By area:

  1. 2025CollabLLM: From Passive Responders to Active Collaboratorscited by 124
  2. 2021GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-Supervised Learning and Explicit Policy Injectioncited by 102
  3. 2024Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Followingcited by 102
  4. 2025Flipping the Dialogue: Training and Evaluating User Language Modelscited by 58
  5. 2024RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learningcited by 49
  6. 2022Controllable Dialogue Simulation with In-Context Learningcited by 42
  7. 2022SPACE-3: Unified Dialog Model Pre-training for Task-Oriented Dialog Understanding and Generationcited by 34
  8. 2023LLM-empowered Chatbots for Psychiatrist and Patient Simulation: Application and Evaluationcited by 34
  9. 2023User Simulation for Evaluating Information Access Systemscited by 31
  10. 2023Think, Act, and Ask: Open-World Interactive Personalized Robot Navigationcited by 31
  11. 2022GenTUS: Simulating User Behaviour and Language in Task-oriented Dialogues with Generative Transformerscited by 23
  12. 2024A Survey on Recent Advances in Conversational Data Generationcited by 23
24 more
  1. 2024Position: Towards Bidirectional Human-AI Alignmentcited by 18
  2. 2022Is MultiWOZ a Solved Task? An Interactive TOD Evaluation Framework with User Simulatorcited by 15
  3. 2022Privacy-Preserving Domain Adaptation of Semantic Parserscited by 12
  4. 2022Toward Self-learning End-to-End Task-Oriented Dialog Systemscited by 9
  5. 2022Metaphorical User Simulators for Evaluating Task-oriented Dialogue Systemscited by 9
  6. 2022Augmenting Multi-Turn Text-to-SQL Datasets with Self-Playcited by 9
  7. 2022KRLS: Improving End-to-End Response Generation in Task Oriented Dialog with Reinforced Keywords Learningcited by 9
  8. 2023EmoUS: Simulating User Emotions in Task-Oriented Dialoguescited by 9
  9. 2022Addressing Resource and Privacy Constraints in Semantic Parsing Through Data Augmentationcited by 7
  10. 2024Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Usecited by 7
  11. 2022A Generative User Simulator with GPT-based Architecture and Goal State Tracking for Reinforced Multi-Domain Dialog Systemscited by 6
  12. 2022Post-processing Networks: Method for Optimizing Pipeline Task-oriented Dialogue Systems using Reinforcement Learningcited by 5
  13. 2024DFA-RAG: Conversational Semantic Router for Large Language Model with Definite Finite Automatoncited by 5
  14. 2023OmniDialog: An Omnipotent Pre-training Model for Task-Oriented Dialogue Systemcited by 4
  15. 2022Dialogue Evaluation with Offline Reinforcement Learningcited by 3
  16. 2023Data Augmentation for Conversational AIcited by 3
  17. 2023Adversarial learning of neural user simulators for dialogue policy optimisationcited by 2
  18. 2024From Intents to Conversations: Generating Intent-Driven Dialogues with Contrastive Learning for Multi-Turn Classificationcited by 2
  19. 2025DialogueForge: LLM Simulation of Human-Chatbot Dialoguecited by 2
  20. 2022UBARv2: Towards Mitigating Exposure Bias in Task-Oriented Dialogscited by 1
  21. 2022Jointly Reinforced User Simulator and Task-oriented Dialog System with Simplified Generative Architecturecited by 0
  22. 2023Dataflow Dialogue Generationcited by 0
  23. 2026From Static Context to Calibrated Interactive RL: Mitigating Distribution Shift in Multi-turn Dialogue with Aligned Simulatorcited by 0
  24. 2026Training User Simulators with Latent Intrinsic Feedbackcited by 0