mlsanityData to 7 Oct 2026

Post-processing Networks: Method for Optimizing Pipeline Task-oriented Dialogue Systems using Reinforcement Learning

arXiv 2207.12185 ↗First posted 25 Jul 2022cs.CL cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Many studies have proposed methods for optimizing the dialogue performance of an entire pipeline task-oriented dialogue system by jointly training modules in the system using reinforcement learning. However, these methods are limited in that they can only be applied to modules implemented using trainable neural-based methods. To solve this problem, we propose a method for optimizing a pipeline system composed of modules implemented with arbitrary methods for dialogue performance. With our method, neural-based components called post-processing networks (PPNs) are installed inside such a system to post-process the output of each module. All PPNs are updated to improve the overall dialogue performance of the system by using reinforcement learning, not necessitating each module to be differentiable. Through dialogue simulation and human evaluation on the MultiWOZ dataset, we show that our method can improve the dialogue performance of pipeline systems consisting of various modules.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 33 of its 40 references that are in the map. First posted on arXiv 25 Jul 2022.

Builds on

33 of 40 references.

References · 33

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  6. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  7. 2018MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modellingcited by 760
  8. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  9. 2016A Network-based End-to-End Trainable Task-oriented Dialogue Systemcited by 252
  10. 2019Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systemscited by 215
  11. 2019BERT for Joint Intent Classification and Slot Fillingcited by 195
  12. 2020Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPOcited by 184
21 more
  1. 2019The Second Conversational Intelligence Challenge (ConvAI2)cited by 178
  2. 2018Neural Approaches to Conversational AIcited by 164
  3. 2017A Survey on Dialogue Systems: Recent Advances and New Frontierscited by 162
  4. 2015Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systemscited by 134
  5. 2019Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Contextcited by 99
  6. 2020Recent Advances and Challenges in Task-oriented Dialog Systemcited by 78
  7. 2018Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systemscited by 53
  8. 2019Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Modelscited by 53
  9. 2017End-to-End Task-Completion Neural Dialogue Systemscited by 52
  10. 2020ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systemscited by 52
  11. 2020Overview of the Ninth Dialog System Technology Challenge: DSTC9cited by 49
  12. 2019Structured Fusion Networks for Dialogcited by 40
  13. 2019Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialogcited by 37
  14. 2021Transferable Dialogue Systems and User Simulatorscited by 35
  15. 2016Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learningcited by 29
  16. 2017End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learningcited by 19
  17. 2019MOSS: End-to-End Dialog System Framework with Modular Supervisioncited by 19
  18. 2020Is Your Goal-Oriented Dialog Model Performing Really Well? Empirical Analysis of System-wise Evaluationcited by 19
  19. 2021Imperfect also Deserves Reward: Multi-Level and Sequential Reward Modeling for Better Dialog Managementcited by 6
  20. 2020SUMBT+LaRL: Effective Multi-domain End-to-end Neural Task-oriented Dialog Systemcited by 3
  21. 2021Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog Systemcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 2; 2024: 1.

5 papers.

  1. 2024Sparse Rewards Can Self-Train Dialogue Agentscited by 6
  2. 2023InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systemscited by 5
  3. 2022Adaptive Natural Language Generation for Task-oriented Dialogue via Reinforcement Learningcited by 2
  4. 2023Rescue Conversations from Dead-ends: Efficient Exploration for Task-oriented Dialogue Policy Optimizationcited by 1
  5. 2022Team Flow at DRC2022: Pipeline System for Travel Destination Recommendation Task in Spoken Dialoguecited by 0