# Inconsistent dialogue responses and how to recover from them

arXiv: https://arxiv.org/abs/2401.10353 · HTML: https://mlsanity.com/paper/2401.10353/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 37 references that are in the map.
- First posted on arXiv 18 Jan 2024.

## Builds on (most cited first, 27 of 27)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Personalizing Dialogue Agents: I have a dog, do you have pets too?](https://mlsanity.com/paper/1801.07243/) (2018)
- [Recipes for building an open-domain chatbot](https://mlsanity.com/paper/2004.13637/) (2020)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [Factuality Enhanced Language Models for Open-Ended Text Generation](https://mlsanity.com/paper/2206.04624/) (2022)
- [Dialogue Natural Language Inference](https://mlsanity.com/paper/1811.00671/) (2018)
- [Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Training](https://mlsanity.com/paper/1911.03860/) (2019)
- [OCNLI: Original Chinese Natural Language Inference](https://mlsanity.com/paper/2010.05444/) (2020)
- [CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation](https://mlsanity.com/paper/2109.05729/) (2021)
- [A Large-Scale Chinese Short-Text Conversation Dataset](https://mlsanity.com/paper/2008.03946/) (2020)
- [ChatGPT or Grammarly? Evaluating ChatGPT on Grammatical Error Correction Benchmark](https://mlsanity.com/paper/2303.13648/) (2023)
- [Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Features](https://mlsanity.com/paper/2107.06963/) (2021)
- [I like fish, especially dolphins: Addressing Contradictions in Dialogue Modeling](https://mlsanity.com/paper/2012.13391/) (2020)
- [UER: An Open-Source Toolkit for Pre-training Models](https://mlsanity.com/paper/1909.05658/) (2019)
- [Contrastive Search Is What You Need For Neural Text Generation](https://mlsanity.com/paper/2210.14140/) (2022)
- [Evaluating Coherence in Dialogue Systems using Entailment](https://mlsanity.com/paper/1904.03371/) (2019)
- [NaturalConv: A Chinese Dialogue Dataset Towards Multi-turn Topic-driven Conversation](https://mlsanity.com/paper/2103.02548/) (2021)
- [Rome was built in 1776: A Case Study on Factual Correctness in Knowledge-Grounded Response Generation](https://mlsanity.com/paper/2110.05456/) (2021)
- [ConjNLI: Natural Language Inference Over Conjunctive Sentences](https://mlsanity.com/paper/2010.10418/) (2020)
- [Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identity](https://mlsanity.com/paper/2112.05843/) (2021)
- [Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System](https://mlsanity.com/paper/2109.11292/) (2021)
- [CDConv: A Benchmark for Contradiction Detection in Chinese Conversations](https://mlsanity.com/paper/2210.08511/) (2022)

## Built on by (most cited first, 3 of 3)

- [Consistency in Language Models: Current Landscape, Challenges, and Future Directions](https://mlsanity.com/paper/2505.00268/) (2025)
- [ReIn: Conversational Error Recovery with Reasoning Inception](https://mlsanity.com/paper/2602.17022/) (2026)
- [Towards Proactive Detection of User-Side Implicit Conflicts in Human-LLM Dialogue](https://mlsanity.com/paper/2609.19155/) (2026)

## Abstract (arXiv)

One critical issue for chat systems is to stay consistent about preferences, opinions, beliefs and facts of itself, which has been shown a difficult problem. In this work, we study methods to assess and bolster utterance consistency of chat systems. A dataset is first developed for studying the inconsistencies, where inconsistent dialogue responses, explanations of the inconsistencies, and recovery utterances are authored by annotators. This covers the life span of inconsistencies, namely introduction, understanding, and resolution. Building on this, we introduce a set of tasks centered on dialogue consistency, specifically focused on its detection and resolution. Our experimental findings indicate that our dataset significantly helps the progress in identifying and resolving conversational inconsistencies, and current popular large language models like ChatGPT which are good at resolving inconsistencies however still struggle with detection.