# Reinforcement Learning of Multi-Domain Dialog Policies Via Action Embeddings

arXiv: https://arxiv.org/abs/2207.00468 · HTML: https://mlsanity.com/paper/2207.00468/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 1 Jul 2022.

## Builds on (most cited first, 24 of 24)

- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Neural Approaches to Conversational AI](https://mlsanity.com/paper/1809.08267/) (2018)
- [A Survey on Dialogue Systems: Recent Advances and New Frontiers](https://mlsanity.com/paper/1711.01731/) (2017)
- [Deep Reinforcement Learning in Large Discrete Action Spaces](https://mlsanity.com/paper/1512.07679/) (2015)
- [Learning End-to-End Goal-Oriented Dialog](https://mlsanity.com/paper/1605.07683/) (2016)
- [Deep Reinforcement Learning with a Natural Language Action Space](https://mlsanity.com/paper/1511.04636/) (2015)
- [Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning](https://mlsanity.com/paper/1704.03084/) (2017)
- [Domain Adaptive Dialog Generation via Meta Learning](https://mlsanity.com/paper/1906.03520/) (2019)
- [Robust Zero-Shot Cross-Domain Slot Filling with Example Values](https://mlsanity.com/paper/1906.06870/) (2019)
- [Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning](https://mlsanity.com/paper/1702.03274/) (2017)
- [BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1711.05715/) (2017)
- [Towards End-to-End Learning for Dialog State Tracking and Management using Deep Reinforcement Learning](https://mlsanity.com/paper/1606.02560/) (2016)
- [End-to-End Offline Goal-Oriented Dialog Policy Learning via Policy Gradient](https://mlsanity.com/paper/1712.02838/) (2017)
- [Zero-Shot Dialog Generation with Cross-Domain Latent Actions](https://mlsanity.com/paper/1805.04803/) (2018)
- [Scalable Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/1712.10224/) (2017)
- [End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning](https://mlsanity.com/paper/1711.10712/) (2017)
- [Toward Continual Learning for Conversational Agents](https://mlsanity.com/paper/1712.09943/) (2017)
- [Subgoal Discovery for Hierarchical Dialogue Policy Learning](https://mlsanity.com/paper/1804.07855/) (2018)
- [Sub-domain Modelling for Dialogue Management with Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1706.06210/) (2017)
- [Multi-Domain Adversarial Learning for Slot Filling in Spoken Language Understanding](https://mlsanity.com/paper/1711.11310/) (2017)
- [Learning Robust Dialog Policies in Noisy Environments](https://mlsanity.com/paper/1712.04034/) (2017)
- [Few-Shot Generalization Across Dialogue Tasks](https://mlsanity.com/paper/1811.11707/) (2018)
- [Dialogue manager domain adaptation using Gaussian process reinforcement learning](https://mlsanity.com/paper/1609.02846/) (2016)
- [Deep Reinforcement Learning for Multi-Domain Dialogue Systems](https://mlsanity.com/paper/1611.08675/) (2016)

## Abstract (arXiv)

Learning task-oriented dialog policies via reinforcement learning typically requires large amounts of interaction with users, which in practice renders such methods unusable for real-world applications. In order to reduce the data requirements, we propose to leverage data from across different dialog domains, thereby reducing the amount of data required from each given domain. In particular, we propose to learn domain-agnostic action embeddings, which capture general-purpose structure that informs the system how to act given the current dialog context, and are then specialized to a specific domain. We show how this approach is capable of learning with significantly less interaction with users, with a reduction of 35% in the number of dialogs required to learn, and to a higher level of proficiency than training separate policies for each domain on a set of simulated domains.