# Measuring Semantic Coherence of a Conversation

arXiv: https://arxiv.org/abs/1806.06411 · HTML: https://mlsanity.com/paper/1806.06411/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2018.

## Built on by (most cited first, 10 of 10)

- [CTRLEval: An Unsupervised Reference-Free Metric for Evaluating Controlled Text Generation](https://mlsanity.com/paper/2204.00862/) (2022)
- [Learning to Relate to Previous Turns in Conversational Search](https://mlsanity.com/paper/2306.02553/) (2023)
- [DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations](https://mlsanity.com/paper/2203.09711/) (2022)
- [Conversational Entity Linking: Problem Definition and Datasets](https://mlsanity.com/paper/2105.04903/) (2021)
- [CausalScore: An Automatic Reference-Free Metric for Assessing Response Relevance in Open-Domain Dialogue Systems](https://mlsanity.com/paper/2406.17300/) (2024)
- [Toward More Accurate and Generalizable Evaluation Metrics for Task-Oriented Dialogs](https://mlsanity.com/paper/2306.03984/) (2023)
- [Personality Differences Drive Conversational Dynamics: A High-Dimensional NLP Approach](https://mlsanity.com/paper/2410.11043/) (2024)
- [Open-Domain Text Evaluation via Contrastive Distribution Methods](https://mlsanity.com/paper/2306.11879/) (2023)
- [Multi-agent AI systems outperform human teams in creativity](https://mlsanity.com/paper/2605.17885/) (2026)
- [Does Linguistic Structure Enrichment Enhance Coherence Assessment? Not With Current Architectures](https://mlsanity.com/paper/2609.10893/) (2026)

## Abstract (arXiv)

Conversational systems have become increasingly popular as a way for humans to interact with computers. To be able to provide intelligent responses, conversational systems must correctly model the structure and semantics of a conversation. We introduce the task of measuring semantic (in)coherence in a conversation with respect to background knowledge, which relies on the identification of semantic relations between concepts introduced during a conversation. We propose and evaluate graph-based and machine learning-based approaches for measuring semantic coherence using knowledge graphs, their vector space embeddings and word embedding models, as sources of background knowledge. We demonstrate how these approaches are able to uncover different coherence patterns in conversations on the Ubuntu Dialogue Corpus.