# The Problem of Coherence in Natural Language Explanations of Recommendations

arXiv: https://arxiv.org/abs/2312.11356 · HTML: https://mlsanity.com/paper/2312.11356/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 33 references that are in the map.
- First posted on arXiv 18 Dec 2023.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [A Contrastive Framework for Neural Text Generation](https://mlsanity.com/paper/2202.06417/) (2022)
- [Explainable Recommendation: A Survey and New Perspectives](https://mlsanity.com/paper/1804.11192/) (2018)
- [Towards Explainable Artificial Intelligence](https://mlsanity.com/paper/1909.12072/) (2019)
- [RUBER: An Unsupervised Method for Automatic Evaluation of Open-Domain Dialog Systems](https://mlsanity.com/paper/1701.03079/) (2017)
- [Personalized Prompt Learning for Explainable Recommendation](https://mlsanity.com/paper/2202.07371/) (2022)
- [Personalized Transformer for Explainable Recommendation](https://mlsanity.com/paper/2105.11601/) (2021)
- [Validating Label Consistency in NER Data Annotation](https://mlsanity.com/paper/2101.08698/) (2021)
- [On Faithfulness and Coherence of Language Explanations for Recommendation Systems](https://mlsanity.com/paper/2209.05409/) (2022)

## Built on by (most cited first, 4 of 4)

- [Faithful and Plausible Natural Language Explanations for Image Classification: A Pipeline Approach](https://mlsanity.com/paper/2407.20899/) (2024)
- [Text Chunking for Document Classification for Urban System Management using Large Language Models](https://mlsanity.com/paper/2504.00274/) (2025)
- [ELIXIR: Efficient and LIghtweight model for eXplaIning Recommendations](https://mlsanity.com/paper/2508.20312/) (2025)
- [MMP-Refer: Multimodal Path Retrieval-augmented LLMs For Explainable Recommendation](https://mlsanity.com/paper/2604.03666/) (2026)

## Abstract (arXiv)

Providing natural language explanations for recommendations is particularly useful from the perspective of a non-expert user. Although several methods for providing such explanations have recently been proposed, we argue that an important aspect of explanation quality has been overlooked in their experimental evaluation. Specifically, the coherence between generated text and predicted rating, which is a necessary condition for an explanation to be useful, is not properly captured by currently used evaluation measures. In this paper, we highlight the issue of explanation and prediction coherence by 1) presenting results from a manual verification of explanations generated by one of the state-of-the-art approaches 2) proposing a method of automatic coherence evaluation 3) introducing a new transformer-based method that aims to produce more coherent explanations than the state-of-the-art approaches 4) performing an experimental evaluation which demonstrates that this method significantly improves the explanation coherence without affecting the other aspects of recommendation performance.