mlsanityData to 7 Oct 2026

Jointly Reinforced User Simulator and Task-oriented Dialog System with Simplified Generative Architecture

arXiv 2210.06706 ↗First posted 13 Oct 2022cs.CL cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recently, there has been progress in supervised funetuning pretrained GPT-2 to build end-to-end task-oriented dialog (TOD) systems. However, online reinforcement learning of a GPT-2 based dialog system (DS), together with a end-to-end user simulator (US), has not ever been explored. Moreover, a drawback with existing GPT-2 based TOD systems is that they mostly employ the whole dialog history as input, which brings inefficiencies in memory and compute. In this paper, we first propose Simplified Generative Architectures (SGA) for DS and US respectively, both based on GPT-2 but using shortened history. Then, we successfully develop Jointly Reinforced US and DS, called SGA-JRUD. Our DS with the proposed SGA, when only supervised trained, achieves state-of-the-art performance on MultiWOZ2.1 and is more compute-efficient in both training and generation. Extensive experiments on MultiWOZ2.1 further show the superiority of SGA-JRUD in both offline and online evaluations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 24 of its 47 references that are in the map. First posted on arXiv 13 Oct 2022.

Builds on

24 of 47 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  3. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  4. 2020A Simple Language Model for Task-Oriented Dialoguecited by 331
  5. 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
  6. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 202
  7. 2016Neural Belief Tracker: Data-Driven Dialogue State Trackingcited by 142
  8. 2020UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2cited by 111
  9. 2019Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Contextcited by 99
  10. 2017Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialogcited by 86
  11. 2019Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Modelscited by 53
  12. 2021Shades of BLEU, Flavours of Success: The Case of MultiWOZcited by 48
12 more
  1. 2020A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learningcited by 47
  2. 2019How to Build User Simulators to Train RL-based Dialog Systemscited by 44
  3. 2018Neural User Simulation for Corpus-based Policy Optimisation for Spoken Dialogue Systemscited by 42
  4. 2018User Modeling for Task Oriented Dialoguescited by 36
  5. 2021Transferable Dialogue Systems and User Simulatorscited by 35
  6. 2017Iterative Policy Learning in End-to-End Trainable Task-Oriented Neural Dialog Modelscited by 26
  7. 2017End-to-End Offline Goal-Oriented Dialog Policy Learning via Policy Gradientcited by 23
  8. 2020Multi-Agent Task-Oriented Dialog Policy Learning with Role-Aware Reward Decompositioncited by 23
  9. 2017An End-to-End Trainable Neural Network Model with Belief Tracking for Task-Oriented Dialogcited by 17
  10. 2017Latent Intention Dialogue Modelscited by 15
  11. 2019Collaborative Multi-Agent Dialogue Model Training Via Reinforcement Learningcited by 9
  12. 2021DORA: Toward Policy Optimization for Task-oriented Dialogue System with Efficient Contextcited by 2

Built on by

Not cited yet.