mlsanityData to 14 Oct 2026

Do LLMs Make More Mistakes If They Do Not Believe the Input Data?

arXiv 2609.09363 ↗First posted 8 Sep 2026cs.CLcited by 0

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Large language models (LLMs) are prone to hallucinating or misinterpreting facts, which impairs their usability in retrieval-augmented generation or data-to-text systems. We analyse how faithfulness of LLMs to provided context depends on how plausible they perceive the context to be (context-memory conflict). To better identify error patterns, we make use of the increased difficulty of non-English and low-resource language text generation and input data based on local knowledge, only partially captured in models' parametric knowledge. We let the models generate text in English, Czech, Slovak and Upper Sorbian from factual (FA), counterfactual (CFA) and fictional (FI) RDF triples containing local Czech and Slovak data. Contrary to our expectations, we observe only a weak context-memory conflict on the human-annotated sample. For Kimi K3 as an LLM judge, which agrees well with human annotations on the sample, counterfactual inputs receive only slightly lower faithfulness scores than factual ones (-0.05 on a 1-5 scale). We also find that a suboptimal choice of LLM judge would lead to overestimating the strength of the context-memory conflict.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 9 of its 35 references that are in the map. First posted on arXiv 8 Sep 2026.

Builds on

9 of 35 references.

References · 9

  1. 2024Knowledge Conflicts for LLMs: A Surveycited by 283
  2. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 243
  3. 2025MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languagescited by 26
  4. 2023Using Large Language Models for Zero-Shot Natural Language Generation from Knowledge Graphscited by 24
  5. 2024Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLMs on Data-to-Text Generationcited by 16
  6. 2025Task Matters: Knowledge Requirements Shape LLM Responses to Context-Memory Conflictcited by 16
  7. 2026Tiny Aya: Bridging Scale and Multilingual Depthcited by 15
  8. 2024High-quality Data-to-Text Generation for Severely Under-Resourced Languages with Out-of-the-box Large Language Modelscited by 5
  9. 2025CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Datasetcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Investigating Factuality in Long-Form Text Generation: The Roles of Self-Known and Self-Unknown
  2. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity
  3. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence
  4. 2025Are the Hidden States Hiding Something? Testing the Limits of Factuality-Encoding Capabilities in LLMs
  5. 2023LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond
  6. 2026Context Shapes LLMs Retrieval-Augmented Fact-Checking Effectiveness
  7. 2024Under the Surface: Tracking the Artifactuality of LLM-Generated Data
  8. 2024To Know or Not To Know? Analyzing Self-Consistency of Large Language Models under Ambiguity