# Open-Domain Dialogue Quality Evaluation: Deriving Nugget-level Scores from Turn-level Scores

arXiv: https://arxiv.org/abs/2310.00410 · HTML: https://mlsanity.com/paper/2310.00410/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 30 Sep 2023.

## Builds on (most cited first, 14 of 14)

- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [What makes a good conversation? How controllable attributes affect human judgments](https://mlsanity.com/paper/1902.08654/) (2019)
- [Unsupervised Evaluation of Interactive Dialog with DialoGPT](https://mlsanity.com/paper/2006.12719/) (2020)
- [Survey on Evaluation Methods for Dialogue Systems](https://mlsanity.com/paper/1905.04071/) (2019)
- [Conversational Recommender System](https://mlsanity.com/paper/1806.03277/) (2018)
- [A Comprehensive Assessment of Dialog Evaluation Metrics](https://mlsanity.com/paper/2106.03706/) (2021)
- [DynaEval: Unifying Turn and Dialogue Level Evaluation](https://mlsanity.com/paper/2106.01112/) (2021)
- [Deconstruct to Reconstruct a Configurable Evaluation Metric for Open-Domain Dialogue Systems](https://mlsanity.com/paper/2011.00483/) (2020)
- [Predictive Engagement: An Efficient Metric For Automatic Evaluation of Open-Domain Dialogue Systems](https://mlsanity.com/paper/1911.01456/) (2019)
- [Towards Coherent and Engaging Spoken Dialog Response Generation Using Automatic Conversation Evaluators](https://mlsanity.com/paper/1904.13015/) (2019)
- [Generating Empathetic Responses by Looking Ahead the User's Sentiment](https://mlsanity.com/paper/1906.08487/) (2019)
- [EnDex: Evaluation of Dialogue Engagingness at Scale](https://mlsanity.com/paper/2210.12362/) (2022)
- [SWAN: A Generic Framework for Auditing Textual Conversational Systems](https://mlsanity.com/paper/2305.08290/) (2023)
- [SelF-Eval: Self-supervised Fine-grained Dialogue Evaluation](https://mlsanity.com/paper/2208.08094/) (2022)

## Built on by (most cited first, 1 of 1)

- [Conversational Gold: Evaluating Personalized Conversational Search System using Gold Nuggets](https://mlsanity.com/paper/2503.09902/) (2025)

## Abstract (arXiv)

Existing dialogue quality evaluation systems can return a score for a given system turn from a particular viewpoint, e.g., engagingness. However, to improve dialogue systems by locating exactly where in a system turn potential problems lie, a more fine-grained evaluation may be necessary. We therefore propose an evaluation approach where a turn is decomposed into nuggets (i.e., expressions associated with a dialogue act), and nugget-level evaluation is enabled by leveraging an existing turn-level evaluation system. We demonstrate the potential effectiveness of our evaluation method through a case study.