Inconsistent dialogue responses and how to recover from them
Not on the map. Why?
Abstract (arXiv)
One critical issue for chat systems is to stay consistent about preferences, opinions, beliefs and facts of itself, which has been shown a difficult problem. In this work, we study methods to assess and bolster utterance consistency of chat systems. A dataset is first developed for studying the inconsistencies, where inconsistent dialogue responses, explanations of the inconsistencies, and recovery utterances are authored by annotators. This covers the life span of inconsistencies, namely introduction, understanding, and resolution. Building on this, we introduce a set of tasks centered on dialogue consistency, specifically focused on its detection and resolution. Our experimental findings indicate that our dataset significantly helps the progress in identifying and resolving conversational inconsistencies, and current popular large language models like ChatGPT which are good at resolving inconsistencies however still struggle with detection.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 18 Jan 2024.
Builds on
27 of 37 references.
References · 27
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2019The Curious Case of Neural Text Degenerationcited by 2,916
- 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
- 2020Recipes for building an open-domain chatbotcited by 593
- 2019Neural Text Generation with Unlikelihood Trainingcited by 503
- 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 202
- 2018Dialogue Natural Language Inferencecited by 135
- 2019Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Trainingcited by 110
15 more
- 2020OCNLI: Original Chinese Natural Language Inferencecited by 86
- 2021CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generationcited by 81
- 2020A Large-Scale Chinese Short-Text Conversation Datasetcited by 77
- 2023ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmarkcited by 77
- 2021Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Featurescited by 74
- 2020I like fish, especially dolphins: Addressing Contradictions in Dialogue Modelingcited by 68
- 2019UER: An Open-Source Toolkit for Pre-training Modelscited by 62
- 2022Contrastive Search Is What You Need For Neural Text Generationcited by 59
- 2019Evaluating Coherence in Dialogue Systems using Entailmentcited by 53
- 2021NaturalConv: A Chinese Dialogue Dataset Towards Multi-turn Topic-driven Conversationcited by 35
- 2021Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generationcited by 29
- 2020ConjNLI: Natural Language Inference Over Conjunctive Sentencescited by 27
- 2021Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identitycited by 22
- 2021Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue Systemcited by 18
- 2022CDConv: A Benchmark for Contradiction Detection in Chinese Conversationscited by 10
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.