mlsanityData to 7 Oct 2026

TrustMargin: Training-Free Arbitration between Parametric Memory and Retrieved Evidence in Large Language Models

arXiv 2606.08397 ↗First posted 7 Jun 2026cs.CL cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models answer knowledge-intensive questions using both parametric memory and retrieved evidence, but neither source is uniformly reliable. Retrieval can fill knowledge gaps, yet distracting passages may override correct closed-book answers. We study this post-generation conflict as answer-level source arbitration: given Direct and RAG answers from the same frozen model, decide which source to trust. We propose TRUSTMARGIN, a training-free, plug-and-play arbitration layer that scores the two existing candidates with the model's own likelihoods. It combines a parametric-prior margin, which tests whether memory accepts the retrieved answer, with an evidence-binding margin, which discounts passage-only salience and measures question-specific support. TRUSTMARGIN selects between Direct and RAG without fine-tuning, external judges, or additional generation. Across 2WIKIMQA and CWQA with three LLaMA scales, TRUSTMARGIN consistently improves over Direct generation and BM25-RAG, recovers part of the Direct/RAG oracle gap, and generalizes to multiple training-free RAG pipelines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 43 of its 48 references that are in the map. First posted on arXiv 7 Jun 2026.

Builds on

43 of 48 references.

References · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  7. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  8. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  9. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  10. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,126
  11. 2019Language Models as Knowledge Bases?cited by 1,922
  12. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
31 more
  1. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  2. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  3. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  4. 2020Leveraging Passage Retrieval with Generative Models for Open Domain Question Answeringcited by 1,203
  5. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  6. 2022Mass-Editing Memory in a Transformercited by 976
  7. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  8. 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 911
  9. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  10. 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 788
  11. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  12. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  13. 2023In-Context Retrieval-Augmented Language Modelscited by 694
  14. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  15. 2023Active Retrieval Augmented Generationcited by 506
  16. 2021Fast Model Editing at Scalecited by 491
  17. 2022Memory-Based Model Editing at Scalecited by 443
  18. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
  19. 2018The Web as a Knowledge-base for Answering Complex Questionscited by 409
  20. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 364
  21. 2024RAFT: Adapting Language Model to Domain Specific RAGcited by 258
  22. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  23. 2023RA-DIT: Retrieval-Augmented Dual Instruction Tuningcited by 194
  24. 2020Generation-Augmented Retrieval for Open-domain Question Answeringcited by 186
  25. 2024Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Modelscited by 150
  26. 2023Test-Time Training on Nearest Neighbors for Large Language Modelscited by 82
  27. 2024RAT: Retrieval Augmented Thoughts Elicit Context-Aware Reasoning in Long-Horizon Generationcited by 75
  28. 2025Parametric Retrieval Augmented Generationcited by 38
  29. 2024Entropy-Based Decoding for Retrieval-Augmented Large Language Modelscited by 19
  30. 2023RECKONING: Reasoning through Dynamic Knowledge Encodingcited by 17
  31. 2026Decide Then Retrieve: A Training-Free Framework with Uncertainty-Guided Triggering and Dual-Path Retrievalcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Trust or Abstain? A Self-Aware RAG Approach
  2. 2025After Retrieval, Before Generation: Enhancing the Trustworthiness of Large Language Models in Retrieval-Augmented Generation
  3. 2026MemToC: Benchmarking Memory-Tool Conflict Resolution in Large Language Models
  4. 2026Regime Boundary Alignment for Evidence-Gated Question Answering
  5. 2026SURE-RAG: Sufficiency and Uncertainty-Aware Evidence Verification for Selective Retrieval-Augmented Generation
  6. 2026Relevance Is Not Sufficient Evidence: Detecting Evidence Gaps Before Generation in RAG
  7. 2025Rethinking All Evidence: Enhancing Trustworthy Retrieval-Augmented Generation via Conflict-Driven Summarization
  8. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts