# What makes a good conversation? How controllable attributes affect human judgments

arXiv: https://arxiv.org/abs/1902.08654 · HTML: https://mlsanity.com/paper/1902.08654/

## Key facts

- Cited by 140 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Feb 2019.

## Built on by (most cited first, 30 of 140)

- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://mlsanity.com/paper/2105.03023/) (2021)
- [FUDGE: Controlled Text Generation With Future Discriminators](https://mlsanity.com/paper/2104.05218/) (2021)
- [SODA: Million-scale Dialogue Distillation with Social Commonsense Contextualization](https://mlsanity.com/paper/2212.10465/) (2022)
- [RL with KL penalties is better viewed as Bayesian inference](https://mlsanity.com/paper/2205.11275/) (2022)
- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Productivity Assessment of Neural Code Completion](https://mlsanity.com/paper/2205.06537/) (2022)
- [Human-centric Dialog Training via Offline Reinforcement Learning](https://mlsanity.com/paper/2010.05848/) (2020)
- [A Comprehensive Assessment of Dialog Evaluation Metrics](https://mlsanity.com/paper/2106.03706/) (2021)
- [Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations](https://mlsanity.com/paper/2308.11995/) (2023)
- [MISC: A MIxed Strategy-Aware Model Integrating COMET for Emotional Support Conversation](https://mlsanity.com/paper/2203.13560/) (2022)
- [Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model](https://mlsanity.com/paper/2310.09520/) (2023)
- [Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling](https://mlsanity.com/paper/2107.03451/) (2021)
- [Evaluating Attribution in Dialogue Systems: The BEGIN Benchmark](https://mlsanity.com/paper/2105.00071/) (2021)
- [Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach](https://mlsanity.com/paper/2101.07714/) (2021)
- [Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features](https://mlsanity.com/paper/2107.06963/) (2021)
- [Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions](https://mlsanity.com/paper/2109.05794/) (2021)
- [Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents](https://mlsanity.com/paper/2201.04723/) (2022)
- [Compression, Transduction, and Creation: A Unified Framework for Evaluating Natural Language Generation](https://mlsanity.com/paper/2109.06379/) (2021)
- [DynaEval: Unifying Turn and Dialogue Level Evaluation](https://mlsanity.com/paper/2106.01112/) (2021)
- [Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts](https://mlsanity.com/paper/2108.11830/) (2021)
- [LENS: A Learnable Evaluation Metric for Text Simplification](https://mlsanity.com/paper/2212.09739/) (2022)
- [Perturbation CheckLists for Evaluating NLG Evaluation Metrics](https://mlsanity.com/paper/2109.05771/) (2021)
- [Leveraging Large Language Models to Power Chatbots for Collecting User Self-Reported Data](https://mlsanity.com/paper/2301.05843/) (2023)
- [A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluators](https://mlsanity.com/paper/2312.15407/) (2023)
- [Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech](https://mlsanity.com/paper/2106.01625/) (2021)
- [Ask what's missing and what's useful: Improving Clarification Question Generation using Global Knowledge](https://mlsanity.com/paper/2104.06828/) (2021)
- [InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning](https://mlsanity.com/paper/2205.12673/) (2022)
- [Multimodal Dialogue Response Generation](https://mlsanity.com/paper/2110.08515/) (2021)
- [Controlling Conditional Language Models without Catastrophic Forgetting](https://mlsanity.com/paper/2112.00791/) (2021)

## Abstract (arXiv)

A good conversation requires balance -- between simplicity and detail; staying on topic and changing it; asking questions and answering them. Although dialogue agents are commonly evaluated via human judgments of overall quality, the relationship between quality and these individual factors is less well-studied. In this work, we examine two controllable neural text generation methods, conditional training and weighted decoding, in order to control four important attributes for chitchat dialogue: repetition, specificity, response-relatedness and question-asking. We conduct a large-scale human evaluation to measure the effect of these control parameters on multi-turn interactive conversations on the PersonaChat task. We provide a detailed analysis of their relationship to high-level aspects of conversation, and show that by controlling combinations of these variables our models obtain clear improvements in human quality judgments.