# Do LLMs Make More Mistakes If They Do Not Believe the Input Data?

arXiv: https://arxiv.org/abs/2609.09363 · HTML: https://mlsanity.com/paper/2609.09363/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation.
- Builds on 9 of its 35 references that are in the map.
- First posted on arXiv 8 Sep 2026.

## Placed in

- Retrieval-augmented generation / [RAG robustness and adaptive retrieval](https://mlsanity.com/rag/rag-robustness-and-adaptive-retrieval/)

## Builds on (most cited first, 9 of 9)

- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks](https://mlsanity.com/paper/2406.18403/) (2024)
- [MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languages](https://mlsanity.com/paper/2504.10356/) (2025)
- [Using Large Language Models for Zero-Shot Natural Language Generation from Knowledge Graphs](https://mlsanity.com/paper/2307.07312/) (2023)
- [Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generation](https://mlsanity.com/paper/2401.10186/) (2024)
- [Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflict](https://mlsanity.com/paper/2506.06485/) (2025)
- [Tiny Aya: Bridging Scale and Multilingual Depth](https://mlsanity.com/paper/2603.11510/) (2026)
- [High-quality Data-to-Text Generation for Severely Under-Resourced Languages with Out-of-the-box Large Language Models](https://mlsanity.com/paper/2402.12267/) (2024)
- [CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Dataset](https://mlsanity.com/paper/2507.22752/) (2025)

## Abstract (arXiv)

Large language models (LLMs) are prone to hallucinating or misinterpreting facts, which impairs their usability in retrieval-augmented generation or data-to-text systems. We analyse how faithfulness of LLMs to provided context depends on how plausible they perceive the context to be (context-memory conflict). To better identify error patterns, we make use of the increased difficulty of non-English and low-resource language text generation and input data based on local knowledge, only partially captured in models' parametric knowledge. We let the models generate text in English, Czech, Slovak and Upper Sorbian from factual (FA), counterfactual (CFA) and fictional (FI) RDF triples containing local Czech and Slovak data. Contrary to our expectations, we observe only a weak context-memory conflict on the human-annotated sample. For Kimi K3 as an LLM judge, which agrees well with human annotations on the sample, counterfactual inputs receive only slightly lower faithfulness scores than factual ones (-0.05 on a 1-5 scale). We also find that a suboptimal choice of LLM judge would lead to overestimating the strength of the context-memory conflict.