# Joint System-Wise Optimization for Pipeline Goal-Oriented Dialog System

arXiv: https://arxiv.org/abs/2106.04835 · HTML: https://mlsanity.com/paper/2106.04835/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 60 references that are in the map.
- First posted on arXiv 9 Jun 2021.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Tasks](https://mlsanity.com/paper/1901.11196/) (2019)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [Neural Approaches to Conversational AI](https://mlsanity.com/paper/1809.08267/) (2018)
- [Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://mlsanity.com/paper/1508.01745/) (2015)
- [Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context](https://mlsanity.com/paper/1911.10484/) (2019)
- [Recent Advances and Challenges in Task-oriented Dialog System](https://mlsanity.com/paper/2003.07490/) (2020)
- [SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching](https://mlsanity.com/paper/2005.05298/) (2020)
- [SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking](https://mlsanity.com/paper/1907.07421/) (2019)
- [Dialogue Learning with Human Teaching and Feedback in End-to-End Trainable Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1804.06512/) (2018)
- [Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models](https://mlsanity.com/paper/1902.08858/) (2019)
- [ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems](https://mlsanity.com/paper/2002.04793/) (2020)
- [Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning](https://mlsanity.com/paper/1704.03084/) (2017)
- [Guided Dialog Policy Learning: Reward Estimation for Multi-Domain Task-Oriented Dialog](https://mlsanity.com/paper/1908.10719/) (2019)
- [Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue](https://mlsanity.com/paper/1906.07220/) (2019)
- [Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning](https://mlsanity.com/paper/1606.02560/) (2016)
- [Iterative Policy Learning in End-to-End Trainable Task-Oriented Neural Dialog Models](https://mlsanity.com/paper/1709.06136/) (2017)
- [The Eighth Dialog System Technology Challenge](https://mlsanity.com/paper/1911.06394/) (2019)
- [Is Your Goal-Oriented Dialog Model Performing Really Well? Empirical Analysis of System-wise Evaluation](https://mlsanity.com/paper/2005.07362/) (2020)
- [Multi-Task Networks With Universe, Group, and Task Feature Learning](https://mlsanity.com/paper/1907.01791/) (2019)
- [TAVAT: Token-Aware Virtual Adversarial Training for Language Understanding](https://mlsanity.com/paper/2004.14543/) (2020)
- [A Hybrid Task-Oriented Dialog System with Domain and Task Adaptive Pretraining](https://mlsanity.com/paper/2102.04506/) (2021)

## Built on by (most cited first, 1 of 1)

- [Post-processing Networks: Method for Optimizing Pipeline Task-oriented Dialogue Systems using Reinforcement Learning](https://mlsanity.com/paper/2207.12185/) (2022)

## Abstract (arXiv)

Recent work (Takanobu et al., 2020) proposed the system-wise evaluation on dialog systems and found that improvement on individual components (e.g., NLU, policy) in prior work may not necessarily bring benefit to pipeline systems in system-wise evaluation. To improve the system-wise performance, in this paper, we propose new joint system-wise optimization techniques for the pipeline dialog system. First, we propose a new data augmentation approach which automates the labeling process for NLU training. Second, we propose a novel stochastic policy parameterization with Poisson distribution that enables better exploration and offers a principled way to compute policy gradient. Third, we propose a reward bonus to help policy explore successful dialogs. Our approaches outperform the competitive pipeline systems from Takanobu et al. (2020) by big margins of 12% success rate in automatic system-wise evaluation and of 16% success rate in human evaluation on the standard multi-domain benchmark dataset MultiWOZ 2.1, and also outperform the recent state-of-the-art end-to-end trained model from DSTC9.