# Semi-Supervised Dialogue Policy Learning via Stochastic Reward Estimation

arXiv: https://arxiv.org/abs/2005.04379 · HTML: https://mlsanity.com/paper/2005.04379/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2020.

## Built on by (most cited first, 8 of 8)

- [A Survey on Deep Semi-supervised Learning](https://mlsanity.com/paper/2103.00550/) (2021)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need?](https://mlsanity.com/paper/2303.11717/) (2023)
- [A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learning](https://mlsanity.com/paper/2202.13675/) (2022)
- [Meta-Learning with Less Forgetting on Large-Scale Non-Stationary Task Distributions](https://mlsanity.com/paper/2209.01501/) (2022)
- [An Interpretable Neuro-Symbolic Reasoning Framework for Task-Oriented Dialogue Generation](https://mlsanity.com/paper/2203.05843/) (2022)
- ["Think Before You Speak": Improving Multi-Action Dialog Policy by Planning Single-Action Dialogs](https://mlsanity.com/paper/2204.11481/) (2022)
- [Multi-Action Dialog Policy Learning from Logged User Feedback](https://mlsanity.com/paper/2302.13505/) (2023)

## Abstract (arXiv)

Dialogue policy optimization often obtains feedback until task completion in task-oriented dialogue systems. This is insufficient for training intermediate dialogue turns since supervision signals (or rewards) are only provided at the end of dialogues. To address this issue, reward learning has been introduced to learn from state-action pairs of an optimal policy to provide turn-by-turn rewards. This approach requires complete state-action annotations of human-to-human dialogues (i.e., expert demonstrations), which is labor intensive. To overcome this limitation, we propose a novel reward learning approach for semi-supervised policy learning. The proposed approach learns a dynamics model as the reward function which models dialogue progress (i.e., state-action sequences) based on expert demonstrations, either with or without annotations. The dynamics model computes rewards by predicting whether the dialogue progress is consistent with expert demonstrations. We further propose to learn action embeddings for a better generalization of the reward function. The proposed approach outperforms competitive policy learning baselines on MultiWOZ, a benchmark multi-domain dataset.