# Diluted Near-Optimal Expert Demonstrations for Guiding Dialogue Stochastic Policy Optimisation

arXiv: https://arxiv.org/abs/2012.04687 · HTML: https://mlsanity.com/paper/2012.04687/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2020.

## Built on by (most cited first, 3 of 3)

- [What Does The User Want? Information Gain for Hierarchical Dialogue Policy Optimisation](https://mlsanity.com/paper/2109.07129/) (2021)
- [Exploring ReAct Prompting for Task-Oriented Dialogue: Insights and Shortcomings](https://mlsanity.com/paper/2412.01262/) (2024)
- [Graph Neural Network Policies and Imitation Learning for Multi-Domain Task-Oriented Dialogues](https://mlsanity.com/paper/2210.05252/) (2022)

## Abstract (arXiv)

A learning dialogue agent can infer its behaviour from interactions with the users. These interactions can be taken from either human-to-human or human-machine conversations. However, human interactions are scarce and costly, making learning from few interactions essential. One solution to speedup the learning process is to guide the agent's exploration with the help of an expert. We present in this paper several imitation learning strategies for dialogue policy where the guiding expert is a near-optimal handcrafted policy. We incorporate these strategies with state-of-the-art reinforcement learning methods based on Q-learning and actor-critic. We notably propose a randomised exploration policy which allows for a seamless hybridisation of the learned policy and the expert. Our experiments show that our hybridisation strategy outperforms several baselines, and that it can accelerate the learning when facing real humans.