# A Deep Reinforcement Learning Chatbot

arXiv: https://arxiv.org/abs/1709.02349 · HTML: https://mlsanity.com/paper/1709.02349/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Sep 2017.

## Built on by (most cited first, 30 of 36)

- [Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog](https://mlsanity.com/paper/1907.00456/) (2019)
- [MiniLLM: On-Policy Distillation of Large Language Models](https://mlsanity.com/paper/2306.08543/) (2023)
- [Imitation Learning from Imperfect Demonstration](https://mlsanity.com/paper/1901.09387/) (2019)
- [Human-centric Dialog Training via Offline Reinforcement Learning](https://mlsanity.com/paper/2010.05848/) (2020)
- [A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?](https://mlsanity.com/paper/2303.11717/) (2023)
- [Text Generation by Learning from Demonstrations](https://mlsanity.com/paper/2009.07839/) (2020)
- [Civil Rephrases Of Toxic Texts With Self-Supervised Transformers](https://mlsanity.com/paper/2102.05456/) (2021)
- [Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review](https://mlsanity.com/paper/2307.02503/) (2023)
- [Jill Watson: A Virtual Teaching Assistant powered by ChatGPT](https://mlsanity.com/paper/2405.11070/) (2024)
- [APPDIA: A Discourse-aware Transformer-based Style Transfer Model for Offensive Social Media Conversations](https://mlsanity.com/paper/2209.08207/) (2022)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices](https://mlsanity.com/paper/2310.18329/) (2023)
- [A Mixture-of-Expert Approach to RL-based Dialogue Management](https://mlsanity.com/paper/2206.00059/) (2022)
- [Market Concentration Implications of Foundation Models](https://mlsanity.com/paper/2311.01550/) (2023)
- [Building Persona Consistent Dialogue Agents with Offline Reinforcement Learning](https://mlsanity.com/paper/2310.10735/) (2023)
- [Multi-User Chat Assistant (MUCA): a Framework Using LLMs to Facilitate Group Conversations](https://mlsanity.com/paper/2401.04883/) (2024)
- [Semi-Offline Reinforcement Learning for Optimized Text Generation](https://mlsanity.com/paper/2306.09712/) (2023)
- [Reinforcement Learning for Generative AI: A Survey](https://mlsanity.com/paper/2308.14328/) (2023)
- [Echoing: Identity Failures when LLM Agents Talk to Each Other](https://mlsanity.com/paper/2511.09710/) (2025)
- [Conversational Agents: Theory and Applications](https://mlsanity.com/paper/2202.03164/) (2022)
- [Dynamic Planning in Open-Ended Dialogue using Reinforcement Learning](https://mlsanity.com/paper/2208.02294/) (2022)
- [Assessing the efficacy of large language models in generating accurate teacher responses](https://mlsanity.com/paper/2307.04274/) (2023)
- [Automatically Exposing Problems with Neural Dialog Models](https://mlsanity.com/paper/2109.06950/) (2021)
- [Bot Wars Evolved: Orchestrating Competing LLMs in a Counterstrike Against Phone Scams](https://mlsanity.com/paper/2503.07036/) (2025)
- [Put Chatbot into Its Interlocutor's Shoes: New Framework to Learn Chatbot Responding with Intention](https://mlsanity.com/paper/2103.16429/) (2021)
- [Amortized Noisy Channel Neural Machine Translation](https://mlsanity.com/paper/2112.08670/) (2021)
- [A Systematic Evaluation of Response Selection for Open Domain Dialogue](https://mlsanity.com/paper/2208.04379/) (2022)
- [Apollonion: Profile-centric Dialog Agent](https://mlsanity.com/paper/2404.08692/) (2024)
- [Offline Reinforcement Learning for Mixture-of-Expert Dialogue Management](https://mlsanity.com/paper/2302.10850/) (2023)
- [Low-Resource Adaptation of Open-Domain Generative Chatbots](https://mlsanity.com/paper/2108.06329/) (2021)

## Abstract (arXiv)

We present MILABOT: a deep reinforcement learning chatbot developed by the Montreal Institute for Learning Algorithms (MILA) for the Amazon Alexa Prize competition. MILABOT is capable of conversing with humans on popular small talk topics through both speech and text. The system consists of an ensemble of natural language generation and retrieval models, including template-based models, bag-of-words models, sequence-to-sequence neural network and latent variable neural network models. By applying reinforcement learning to crowdsourced data and real-world user interactions, the system has been trained to select an appropriate response from the models in its ensemble. The system has been evaluated through A/B testing with real-world users, where it performed significantly better than many competing systems. Due to its machine learning architecture, the system is likely to improve with additional data.