# When Context Leads but Parametric Memory Follows in Large Language Models

arXiv: https://arxiv.org/abs/2409.08435 · HTML: https://mlsanity.com/paper/2409.08435/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 37 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [Entity-Based Knowledge Conflicts in Question Answering](https://mlsanity.com/paper/2109.05052/) (2021)
- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [Prompting GPT-3 To Be Reliable](https://mlsanity.com/paper/2210.09150/) (2022)
- [Fine-tuning Language Models for Factuality](https://mlsanity.com/paper/2311.08401/) (2023)
- [Large Language Models with Controllable Working Memory](https://mlsanity.com/paper/2211.05110/) (2022)
- [Hurdles to Progress in Long-form Question Answering](https://mlsanity.com/paper/2103.06332/) (2021)
- [Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation](https://mlsanity.com/paper/2212.07981/) (2022)
- [Language Models as Knowledge Bases: On Entity Representations, Storage Capacity, and Paraphrased Queries](https://mlsanity.com/paper/2008.09036/) (2020)
- [DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering](https://mlsanity.com/paper/2211.05655/) (2022)
- [FABLES: Evaluating faithfulness and content selection in book-length summarization](https://mlsanity.com/paper/2404.01261/) (2024)
- [Give Us the Facts: Enhancing Large Language Models with Knowledge Graphs for Fact-aware Language Modeling](https://mlsanity.com/paper/2306.11489/) (2023)
- [Do LLMs Know about Hallucination? An Empirical Investigation of LLM's Hidden States](https://mlsanity.com/paper/2402.09733/) (2024)
- [Knowledge Card: Filling LLMs' Knowledge Gaps with Plug-in Specialized Language Models](https://mlsanity.com/paper/2305.09955/) (2023)
- [Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problem](https://mlsanity.com/paper/2403.03558/) (2024)
- [Prompting Large Language Models for Counterfactual Generation: An Empirical Study](https://mlsanity.com/paper/2305.14791/) (2023)
- [Fine-Grained Natural Language Inference Based Faithfulness Evaluation for Diverse Summarisation Tasks](https://mlsanity.com/paper/2402.17630/) (2024)
- [Does the Generator Mind its Contexts? An Analysis of Generative Model Faithfulness under Context Transfer](https://mlsanity.com/paper/2402.14488/) (2024)

## Built on by (most cited first, 11 of 11)

- [Doc-to-LoRA: Learning to Instantly Internalize Contexts](https://mlsanity.com/paper/2602.15902/) (2026)
- [Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge Enhancement](https://mlsanity.com/paper/2503.23895/) (2025)
- [Memory in Large Language Models: Mechanisms, Evaluation and Evolution](https://mlsanity.com/paper/2509.18868/) (2025)
- [Do Role-Playing Agents Practice What They Preach? Belief-Behavior Consistency in LLM-Based Simulations of Human Trust](https://mlsanity.com/paper/2507.02197/) (2025)
- ["Lost-in-the-Later": Framework for Quantifying Contextual Grounding in Large Language Models](https://mlsanity.com/paper/2507.05424/) (2025)
- [KScope: A Framework for Characterizing the Knowledge Status of Language Models](https://mlsanity.com/paper/2506.07458/) (2025)
- [Multi-Step Knowledge Interaction Analysis via Rank-2 Subspace Disentanglement](https://mlsanity.com/paper/2511.01706/) (2025)
- [Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Retrieval, Reasoning, and Hallucination in Long-Context LLMs](https://mlsanity.com/paper/2601.02023/) (2026)
- [Evaluating the accuracy of KV cache reuse techniques](https://mlsanity.com/paper/2609.31415/) (2026)
- [When Upstream Messages Override Correct Answers: A Controlled Study of Multi-Agent LLM Collaboration](https://mlsanity.com/paper/2609.36855/) (2026)
- [Invisible Ink, Visible Lies: How Production Watermarking Causes LLMs to Hallucinate](https://mlsanity.com/paper/2610.04860/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have demonstrated remarkable progress in leveraging diverse knowledge sources. This study investigates how nine widely used LLMs allocate knowledge between local context and global parameters when answering open-ended questions in knowledge-consistent scenarios. We introduce a novel dataset, WikiAtomic, and systematically vary context sizes to analyze how LLMs prioritize and utilize the provided information and their parametric knowledge in knowledge-consistent scenarios. Additionally, we also study their tendency to hallucinate under varying context sizes. Our findings reveal consistent patterns across models, including a consistent reliance on both contextual (around 70%) and parametric (around 30%) knowledge, and a decrease in hallucinations with increasing context. These insights highlight the importance of more effective context organization and developing models that use input more deterministically for robust performance.