# Two Approaches to Building Collaborative, Task-Oriented Dialog Agents through Self-Play

arXiv: https://arxiv.org/abs/2109.09597 · HTML: https://mlsanity.com/paper/2109.09597/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 27 references that are in the map.
- First posted on arXiv 20 Sep 2021.

## Builds on (most cited first, 12 of 12)

- [Deal or No Deal? End-to-End Learning for Negotiation Dialogues](https://mlsanity.com/paper/1706.05125/) (2017)
- [The Hanabi Challenge: A New Frontier for AI Research](https://mlsanity.com/paper/1902.00506/) (2019)
- [Building a Conversational Agent Overnight with Dialogue Self-Play](https://mlsanity.com/paper/1801.04871/) (2018)
- [RLlib: Abstractions for Distributed Reinforcement Learning](https://mlsanity.com/paper/1712.09381/) (2017)
- [How to Build User Simulators to Train RL-based Dialog Systems](https://mlsanity.com/paper/1909.01388/) (2019)
- [Recommendation as a Communication Game: Self-Supervised Bot-Play for Goal-oriented Dialogue](https://mlsanity.com/paper/1909.03922/) (2019)
- [Neural User Simulation for Corpus-based Policy Optimisation for Spoken Dialogue Systems](https://mlsanity.com/paper/1805.06966/) (2018)
- [A Sequence-to-Sequence Model for User Simulation in Spoken Dialogue Systems](https://mlsanity.com/paper/1607.00070/) (2016)
- [User Modeling for Task Oriented Dialogues](https://mlsanity.com/paper/1811.04369/) (2018)
- [Adversarial Learning of Task-Oriented Neural Dialog Models](https://mlsanity.com/paper/1805.11762/) (2018)
- [Generating Challenge Datasets for Task-Oriented Conversational Agents through Self-Play](https://mlsanity.com/paper/1910.07357/) (2019)
- [MEEP: An Open-Source Platform for Human-Human Dialog Collection and End-to-End Agent Training](https://mlsanity.com/paper/2010.04747/) (2020)

## Abstract (arXiv)

Task-oriented dialog systems are often trained on human/human dialogs, such as collected from Wizard-of-Oz interfaces. However, human/human corpora are frequently too small for supervised training to be effective. This paper investigates two approaches to training agent-bots and user-bots through self-play, in which they autonomously explore an API environment, discovering communication strategies that enable them to solve the task. We give empirical results for both reinforcement learning and game-theoretic equilibrium finding.