# Sub-domain Modelling for Dialogue Management with Hierarchical Reinforcement Learning

arXiv: https://arxiv.org/abs/1706.06210 · HTML: https://mlsanity.com/paper/1706.06210/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Jun 2017.

## Built on by (most cited first, 8 of 8)

- [Neural Approaches to Conversational Information Retrieval](https://mlsanity.com/paper/2201.05176/) (2022)
- [A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learning](https://mlsanity.com/paper/2202.13675/) (2022)
- [Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning](https://mlsanity.com/paper/1801.06176/) (2018)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Graph Neural Network Policies and Imitation Learning for Multi-Domain Task-Oriented Dialogues](https://mlsanity.com/paper/2210.05252/) (2022)
- [Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues](https://mlsanity.com/paper/2302.11199/) (2023)
- [Reinforcement Learning of Multi-Domain Dialog Policies Via Action Embeddings](https://mlsanity.com/paper/2207.00468/) (2022)
- [JoTR: A Joint Transformer and Reinforcement Learning Framework for Dialog Policy Learning](https://mlsanity.com/paper/2309.00230/) (2023)

## Abstract (arXiv)

Human conversation is inherently complex, often spanning many different topics/domains. This makes policy learning for dialogue systems very challenging. Standard flat reinforcement learning methods do not provide an efficient framework for modelling such dialogues. In this paper, we focus on the under-explored problem of multi-domain dialogue management. First, we propose a new method for hierarchical reinforcement learning using the option framework. Next, we show that the proposed architecture learns faster and arrives at a better policy than the existing flat ones do. Moreover, we show how pretrained policies can be adapted to more complex systems with an additional set of new actions. In doing that, we show that our approach has the potential to facilitate policy optimisation for more sophisticated multi-domain dialogue systems.