# End-to-end LSTM-based dialog control optimized with supervised and reinforcement learning

arXiv: https://arxiv.org/abs/1606.01269 · HTML: https://mlsanity.com/paper/1606.01269/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2016.

## Built on by (most cited first, 8 of 8)

- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Schema-Guided Paradigm for Zero-Shot Dialog](https://mlsanity.com/paper/2106.07056/) (2021)
- [Multi-User Chat Assistant (MUCA): a Framework Using LLMs to Facilitate Group Conversations](https://mlsanity.com/paper/2401.04883/) (2024)
- [A Review of Dialogue Systems: From Trained Monkeys to Stochastic Parrots](https://mlsanity.com/paper/2111.01414/) (2021)
- [Bridging UI Design and chatbot Interactions: Applying Form-Based Principles to Conversational Agents](https://mlsanity.com/paper/2507.01862/) (2025)
- [Recent Progress in Conversational AI](https://mlsanity.com/paper/2204.09719/) (2022)
- [Examining the Effectiveness of Chatbots in Gathering Family History Information in Comparison to the Standard In-Person Interview-Based Approach](https://mlsanity.com/paper/2309.03223/) (2023)

## Abstract (arXiv)

This paper presents a model for end-to-end learning of task-oriented dialog systems. The main component of the model is a recurrent neural network (an LSTM), which maps from raw dialog history directly to a distribution over system actions. The LSTM automatically infers a representation of dialog history, which relieves the system developer of much of the manual feature engineering of dialog state. In addition, the developer can provide software that expresses business rules and provides access to programmatic APIs, enabling the LSTM to take actions in the real world on behalf of the user. The LSTM can be optimized using supervised learning (SL), where a domain expert provides example dialogs which the LSTM should imitate; or using reinforcement learning (RL), where the system improves by interacting directly with end users. Experiments show that SL and RL are complementary: SL alone can derive a reasonable initial policy from a small number of training dialogs; and starting RL optimization with a policy trained with SL substantially accelerates the learning rate of RL.