# Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems

arXiv: https://arxiv.org/abs/1906.09308 · HTML: https://mlsanity.com/paper/1906.09308/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jun 2019.

## Built on by (most cited first, 30 of 42)

- [Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog](https://mlsanity.com/paper/1907.00456/) (2019)
- [Advances and Challenges in Conversational Recommender Systems: A Survey](https://mlsanity.com/paper/2101.09459/) (2021)
- [Human-centric Dialog Training via Offline Reinforcement Learning](https://mlsanity.com/paper/2010.05848/) (2020)
- [Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents](https://mlsanity.com/paper/2201.04723/) (2022)
- [DynaEval: Unifying Turn and Dialogue Level Evaluation](https://mlsanity.com/paper/2106.01112/) (2021)
- [Nearly Horizon-Free Offline Reinforcement Learning](https://mlsanity.com/paper/2103.14077/) (2021)
- [The Homogenizing Effect of Large Language Models on Human Expression and Thought](https://mlsanity.com/paper/2508.01491/) (2025)
- [Semantic Diversity in Dialogue with Natural Language Inference](https://mlsanity.com/paper/2205.01497/) (2022)
- [A Literature Survey of Recent Advances in Chatbots](https://mlsanity.com/paper/2201.06657/) (2022)
- [Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue models](https://mlsanity.com/paper/2109.03300/) (2021)
- [TIAGE: A Benchmark for Topic-Shift Aware Dialog Modeling](https://mlsanity.com/paper/2109.04562/) (2021)
- [IQA-EVAL: Automatic Evaluation of Human-Model Interactive Question Answering](https://mlsanity.com/paper/2408.13545/) (2024)
- [Large Language Models Perform Diagnostic Reasoning](https://mlsanity.com/paper/2307.08922/) (2023)
- [Interacting with Non-Cooperative User: A New Paradigm for Proactive Dialogue Policy](https://mlsanity.com/paper/2204.07433/) (2022)
- [Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach](https://mlsanity.com/paper/2102.10242/) (2021)
- [OpenViDial 2.0: A Larger-Scale, Open-Domain Dialogue Generation Dataset with Visual Contexts](https://mlsanity.com/paper/2109.12761/) (2021)
- [A Mixture-of-Expert Approach to RL-based Dialogue Management](https://mlsanity.com/paper/2206.00059/) (2022)
- [On Gap-dependent Bounds for Offline Reinforcement Learning](https://mlsanity.com/paper/2206.00177/) (2022)
- [FineD-Eval: Fine-grained Automatic Dialogue-Level Evaluation](https://mlsanity.com/paper/2210.13832/) (2022)
- [Approximating Online Human Evaluation of Social Chatbots with Prompting](https://mlsanity.com/paper/2304.05253/) (2023)
- [Don't Copy the Teacher: Data and Model Challenges in Embodied Dialogue](https://mlsanity.com/paper/2210.04443/) (2022)
- [A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions](https://mlsanity.com/paper/2504.16939/) (2025)
- [MT-PingEval: Evaluating Multi-Turn Collaboration with Private Information Games](https://mlsanity.com/paper/2602.24188/) (2026)
- [Towards Standard Criteria for human evaluation of Chatbots: A Survey](https://mlsanity.com/paper/2105.11197/) (2021)
- [EnDex: Evaluation of Dialogue Engagingness at Scale](https://mlsanity.com/paper/2210.12362/) (2022)
- [Multi-Granularity Prompts for Topic Shift Detection in Dialogue](https://mlsanity.com/paper/2305.14006/) (2023)
- [Long-term Control for Dialogue Generation: Methods and Evaluation](https://mlsanity.com/paper/2205.07352/) (2022)
- [Topic Shift Detection in Chinese Dialogues: Corpus and Benchmark](https://mlsanity.com/paper/2305.01195/) (2023)
- [clem:todd: A Framework for the Systematic Benchmarking of LLM-Based Task-Oriented Dialogue System Realisations](https://mlsanity.com/paper/2505.05445/) (2025)
- [Achieving Conversational Goals with Unsupervised Post-hoc Knowledge Injection](https://mlsanity.com/paper/2203.11399/) (2022)

## Abstract (arXiv)

Building an open-domain conversational agent is a challenging problem. Current evaluation methods, mostly post-hoc judgments of static conversation, do not capture conversation quality in a realistic interactive context. In this paper, we investigate interactive human evaluation and provide evidence for its necessity; we then introduce a novel, model-agnostic, and dataset-agnostic method to approximate it. In particular, we propose a self-play scenario where the dialog system talks to itself and we calculate a combination of proxies such as sentiment and semantic coherence on the conversation trajectory. We show that this metric is capable of capturing the human-rated quality of a dialog model better than any automated metric known to-date, achieving a significant Pearson correlation (r>.7, p<.05). To investigate the strengths of this novel metric and interactive evaluation in comparison to state-of-the-art metrics and human evaluation of static conversations, we perform extended experiments with a set of models, including several that make novel improvements to recent hierarchical dialog generation architectures through sentiment and semantic knowledge distillation on the utterance level. Finally, we open-source the interactive evaluation platform we built and the dataset we collected to allow researchers to efficiently deploy and evaluate dialog models.