# End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1711.10712 · HTML: https://mlsanity.com/paper/1711.10712/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2017.

## Built on by (most cited first, 19 of 19)

- [Transferable Dialogue Systems and User Simulators](https://mlsanity.com/paper/2107.11904/) (2021)
- [Diaformer: Automatic Diagnosis via Symptoms Sequence Generation](https://mlsanity.com/paper/2112.10433/) (2021)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2311.09008/) (2023)
- [Data Augmentation with Paraphrase Generation and Entity Extraction for Multimodal Dialogue System](https://mlsanity.com/paper/2205.04006/) (2022)
- [Multimodal Conversational AI: A Survey of Datasets and Approaches](https://mlsanity.com/paper/2205.06907/) (2022)
- [Alexa Conversations: An Extensible Data-driven Approach for Building Task-oriented Dialogue Systems](https://mlsanity.com/paper/2104.09088/) (2021)
- [KETOD: Knowledge-Enriched Task-Oriented Dialogue](https://mlsanity.com/paper/2205.05589/) (2022)
- [Conversational Agents: Theory and Applications](https://mlsanity.com/paper/2202.03164/) (2022)
- [SYNDICOM: Improving Conversational Commonsense with Error-Injection and Natural Language Feedback](https://mlsanity.com/paper/2309.10015/) (2023)
- [Post-processing Networks: Method for Optimizing Pipeline Task-oriented Dialogue Systems using Reinforcement Learning](https://mlsanity.com/paper/2207.12185/) (2022)
- [Spoken Conversational Agents with Large Language Models](https://mlsanity.com/paper/2512.02593/) (2025)
- [NLU for Game-based Learning in Real: Initial Evaluations](https://mlsanity.com/paper/2205.13754/) (2022)
- [End-to-End Evaluation of a Spoken Dialogue System for Learning Basic Mathematics](https://mlsanity.com/paper/2211.03511/) (2022)
- [Reinforcement Learning via Auxiliary Task Distillation](https://mlsanity.com/paper/2406.17168/) (2024)
- [BotSIM: An End-to-End Bot Simulation Framework for Commercial Task-Oriented Dialog Systems](https://mlsanity.com/paper/2211.11982/) (2022)
- [Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systems](https://mlsanity.com/paper/2307.07255/) (2023)
- [Building Goal-Oriented Dialogue Systems with Situated Visual Context](https://mlsanity.com/paper/2111.11576/) (2021)
- [Reinforcement Learning of Multi-Domain Dialog Policies Via Action Embeddings](https://mlsanity.com/paper/2207.00468/) (2022)

## Abstract (arXiv)

In this paper, we present a neural network based task-oriented dialogue system that can be optimized end-to-end with deep reinforcement learning (RL). The system is able to track dialogue state, interface with knowledge bases, and incorporate query results into agent's responses to successfully complete task-oriented dialogues. Dialogue policy learning is conducted with a hybrid supervised and deep RL methods. We first train the dialogue agent in a supervised manner by learning directly from task-oriented dialogue corpora, and further optimize it with deep RL during its interaction with users. In the experiments on two different dialogue task domains, our model demonstrates robust performance in tracking dialogue state and producing reasonable system responses. We show that deep RL based optimization leads to significant improvement on task success rate and reduction in dialogue length comparing to supervised training model. We further show benefits of training task-oriented dialogue model end-to-end comparing to component-wise optimization with experiment results on dialogue simulations and human evaluations.