mlsanityData to 14 Oct 2026

Investigating Context-Faithfulness in Large Language Models: The Roles of Memory Strength and Evidence Style

arXiv 2409.10955 ↗First posted 17 Sep 2024cs.CL cs.AIcited by 11

On the map: Retrieval-augmented generation → RAG robustness and adaptive retrieval

Abstract (arXiv)

Retrieval-augmented generation (RAG) improves Large Language Models (LLMs) by incorporating external information into the response generation process. However, how context-faithful LLMs are and what factors influence LLMs' context faithfulness remain largely unexplored. In this study, we investigate the impact of memory strength and evidence presentation on LLMs' receptiveness to external evidence. We quantify the memory strength of LLMs by measuring the divergence in LLMs' responses to different paraphrases of the same question, which is not considered by previous works. We also generate evidence in various styles to examine LLMs' behavior. Our results show that for questions with high memory strength, LLMs are more likely to rely on internal memory. Furthermore, presenting paraphrased evidence significantly increases LLMs' receptiveness compared to simple repetition or adding details. These findings provide key insights for improving retrieval-augmented generation and context-aware LLMs. Our code is available at https://github.com/liyp0095/ContextFaithful.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in RAG robustness and adaptive retrieval, one of 15 directions in Retrieval-augmented generation. Builds on 28 of its 45 references that are in the map. First posted on arXiv 17 Sep 2024.

Builds on

28 of 45 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024From RAGs to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries18 Jun
  2. 2024Tug-of-War Between Knowledge: Exploring and Resolving Knowledge Conflicts in Retrieval-Augmented Language Models22 Feb

Other references · 26

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  3. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  4. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
  5. 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 476
  6. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  7. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 341
  8. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 318
  9. 2022Prompting GPT-3 To Be Reliablecited by 260
  10. 2022RealTime QA: What's the Answer Right Now?cited by 244
  11. 2021Mind the Gap: Assessing Temporal Generalization in Neural Language Modelscited by 230
  12. 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
14 more
  1. 2020How Context Affects Language Models' Factual Predictionscited by 181
  2. 2022Large Language Models with Controllable Working Memorycited by 169
  3. 2022Rich Knowledge Sources Bring Complex Knowledge Conflicts: Recalibrating Models to Reflect Conflicting Evidencecited by 144
  4. 2022DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answeringcited by 96
  5. 2023Retrieving Multimodal Information for Augmented Generation: A Surveycited by 85
  6. 2024What Evidence Do Language Models Find Convincing?cited by 75
  7. 2021BeliefBank: Adding Memory to a Pre-Trained Language Model for a Systematic Notion of Beliefcited by 57
  8. 2023Knowing What LLMs DO NOT Know: A Simple Yet Effective Self-Detection Methodcited by 51
  9. 2024BioRAG: A RAG-LLM Framework for Biological Question Reasoningcited by 43
  10. 2021Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLPcited by 31
  11. 2022Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysiscited by 30
  12. 2022Distantly Supervised Named Entity Recognition via Confidence-Based Multi-Class Positive and Unlabeled Learningcited by 21
  13. 2022Extracting or Guessing? Improving Faithfulness of Event Temporal Relation Extractioncited by 15
  14. 2024Re-Examine Distantly Supervised NER: A New Benchmark and a Simple Approachcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 9.

11 papers.

  1. 2024Context-DPO: Aligning Language Models for Context-Faithfulnesscited by 50
  2. 2026Can LLMs Introspect? A Reality Checkcited by 12
  3. 2026Learning When to Trust via Selective Context Preference Optimizationcited by 3
  4. 2025Copy-Paste to Mitigate Large Language Model Hallucinationscited by 2
  5. 2026Temporal Fact Conflicts in LLMs: Reproducibility Insights from Unifying DYNAMICQA and MULANcited by 2
  6. 2026Beyond Text Following: Repairable Arbitration Reversals in Audio-Language Modelscited by 2
  7. 2026Plans for Evaluating Structured Generative Search Summariescited by 0
  8. 2026CARE-MH: Towards Unified, Reproducible, and Comparable Evaluation of Mental Health LLMscited by 0
  9. 2026DenialRAG: Single-Document RAG Poisoning via Embedded Parametric Denialcited by 0
  10. 2026Rewired or Gated? How Instruction Tuning Shapes Knowledge-Conflict Circuits in LLMscited by 0
  11. 2026Characterizing Overconfident Failure in LLM-Based Code Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidence
  2. 2026Context Shapes LLMs Retrieval-Augmented Fact-Checking Effectiveness
  3. 2024Quantifying reliance on external information over parametric knowledge during Retrieval Augmented Generation (RAG) using mechanistic analysis
  4. 2025LLM-Specific Utility for Retrieval-Augmented Generation
  5. 2026Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG
  6. 2026Beyond the Parameters: A Technical Survey of Contextual Enrichment in Large Language Models: From In-Context Prompting to Causal Retrieval-Augmented Generation
  7. 2023Characterizing Attribution and Fluency Tradeoffs for Retrieval-Augmented Large Language Models
  8. 2026Does More Retrieved Evidence Help Visual Retrieval-Augmented Generation with Diffusion Language Models?