# Learning from Real Users: Rating Dialogue Success with Neural Networks for Reinforcement Learning in Spoken Dialogue Systems

arXiv: https://arxiv.org/abs/1508.03386 · HTML: https://mlsanity.com/paper/1508.03386/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2015.

## Built on by (most cited first, 8 of 8)

- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Shades of BLEU, Flavours of Success: The Case of MultiWOZ](https://mlsanity.com/paper/2106.05555/) (2021)
- [CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learning](https://mlsanity.com/paper/2204.08426/) (2022)
- [A Survey on Recent Advances and Challenges in Reinforcement Learning Methods for Task-Oriented Dialogue Policy Learning](https://mlsanity.com/paper/2202.13675/) (2022)
- [Multi-User Chat Assistant (MUCA): a Framework Using LLMs to Facilitate Group Conversations](https://mlsanity.com/paper/2401.04883/) (2024)
- [Response-act Guided Reinforced Dialogue Generation for Mental Health Counseling](https://mlsanity.com/paper/2301.12729/) (2023)
- [Enhancing End-to-End Multi-Task Dialogue Systems: A Study on Intrinsic Motivation Reinforcement Learning Algorithms for Improved Training and Adaptability](https://mlsanity.com/paper/2401.18040/) (2024)
- [Task-oriented Dialogue Systems: performance vs. quality-optima, a review](https://mlsanity.com/paper/2112.11176/) (2021)

## Abstract (arXiv)

To train a statistical spoken dialogue system (SDS) it is essential that an accurate method for measuring task success is available. To date training has relied on presenting a task to either simulated or paid users and inferring the dialogue's success by observing whether this presented task was achieved or not. Our aim however is to be able to learn from real users acting under their own volition, in which case it is non-trivial to rate the success as any prior knowledge of the task is simply unavailable. User feedback may be utilised but has been found to be inconsistent. Hence, here we present two neural network models that evaluate a sequence of turn-level features to rate the success of a dialogue. Importantly these models make no use of any prior knowledge of the user's task. The models are trained on dialogues generated by a simulated user and the best model is then used to train a policy on-line which is shown to perform at least as well as a baseline system using prior knowledge of the user's task. We note that the models should also be of interest for evaluating SDS and for monitoring a dialogue in rule-based SDS.