mlsanityData to 7 Oct 2026

Quantifying Document Impact in RAG-LLMs

arXiv 2601.05260 ↗First posted 27 Oct 2025cs.IR cs.AI cs.CLcited by 0

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Retrieval Augmented Generation (RAG) enhances Large Language Models (LLMs) by connecting them to external knowledge, improving accuracy and reducing outdated information. However, this introduces challenges such as factual inconsistencies, source conflicts, bias propagation, and security vulnerabilities, which undermine the trustworthiness of RAG systems. A key gap in current RAG evaluation is the lack of a metric to quantify the contribution of individual retrieved documents to the final output. To address this, we introduce the Influence Score (IS), a novel metric based on Partial Information Decomposition that measures the impact of each retrieved document on the generated response. We validate IS through two experiments. First, a poison attack simulation across three datasets demonstrates that IS correctly identifies the malicious document as the most influential in 86% of cases. Second, an ablation study shows that a response generated using only the top-ranked documents by IS is consistently judged more similar to the original response than one generated from the remaining documents. These results confirm the efficacy of IS in isolating and quantifying document influence, offering a valuable tool for improving the transparency and reliability of RAG systems.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 25 of its 28 references that are in the map. First posted on arXiv 27 Oct 2025.

Builds on

25 of 28 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMs13 Jun
  2. 2025CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Models26 May
  3. 2025RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models25 Apr
  4. 2025The RAG Paradox: A Black-Box Attack Exploiting Unintentional Vulnerabilities in Retrieval-Augmented Generation Systems28 Feb
  5. 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models3 Jun
  6. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models12 Feb

Other references · 19

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  4. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  5. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  6. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  7. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  8. 2023Challenges and Applications of Large Language Modelscited by 251
  9. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
  10. 2024RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMscited by 200
  11. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
  12. 2024ClashEval: Quantifying the tug-of-war between an LLM's internal prior and external evidencecited by 109
7 more
  1. 2023RECALL: A Benchmark for LLMs Robustness against External Counterfactual Knowledgecited by 51
  2. 2023Unveiling the Implicit Toxicity in Large Language Modelscited by 51
  3. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 44
  4. 2024Does RAG Introduce Unfairness in LLMs? Evaluating Fairness in Retrieval-Augmented Generation Systemscited by 26
  5. 2024Long Context RAG Performance of Large Language Modelscited by 23
  6. 2019A Novel Approach to the Partial Information Decompositioncited by 16
  7. 2025Mitigating Bias in RAG: Controlling the Embeddercited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025RAG Security and Privacy: Formalizing the Threat Model and Attack Surface
  2. 2026Influence Factors on RAG Poisoning
  3. 2025RAGPart & RAGMask: Retrieval-Stage Defenses Against Corpus Poisoning in Retrieval-Augmented Generation
  4. 2024Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations
  5. 2024Understanding the Impact of Confidence in Retrieval Augmented Generation: A Case Study in the Medical Domain
  6. 2026RAG-Safety-Bench: Reliable Evaluation of Retrieval-Augmented LLM Safety
  7. 2024On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
  8. 2026Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation