mlsanityData to 14 Oct 2026

Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generation

arXiv 2508.02835 ↗First posted 4 Aug 2025cs.LG cs.IRcited by 11

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) has emerged as a powerful approach to boost the capabilities of large language models (LLMs) by incorporating external, up-to-date knowledge sources. However, this introduces a potential vulnerability to knowledge poisoning attacks, where attackers can compromise the knowledge source to mislead the generation model. One such attack is the PoisonedRAG in which the injected adversarial texts steer the model to generate an attacker-chosen response to a target question. In this work, we propose novel defense methods, FilterRAG and ML-FilterRAG, to mitigate the PoisonedRAG attack. First, we propose a new property to uncover distinct properties to differentiate between adversarial and clean texts in the knowledge data source. Next, we employ this property to filter out adversarial texts from clean ones in the design of our proposed approaches. Evaluation of these methods using benchmark datasets demonstrate their effectiveness, with performances close to those of the original RAG systems.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 29 of its 34 references that are in the map. First posted on arXiv 4 Aug 2025.

Builds on

29 of 34 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Stealthy LLM-Driven Data Poisoning Attacks Against Embedding-Based Retrieval-Augmented Recommender Systems8 May
  2. 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Models3 Feb
  3. 2024Certifiably Robust RAG against Retrieval Corruption24 May
  4. 2024Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations22 Apr
  5. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models12 Feb

Other references · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2024The Llama 3 Herd of Modelscited by 14,715
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2024GPT-4o System Cardcited by 4,588
  8. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  9. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  10. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  11. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  12. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
12 more
  1. 2021Improving language models by retrieving from trillions of tokenscited by 1,293
  2. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  3. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  4. 2023PaLM 2 Technical Reportcited by 1,162
  5. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 849
  6. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 159
  7. 2023AnglE-optimized Text Embeddingscited by 99
  8. 2023WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipediacited by 67
  9. 2020Frequency-Guided Word Substitutions for Detecting Textual Adversarial Examplescited by 54
  10. 2020BERT-kNN: Adding a kNN Search Component to Pretrained Language Models for Better QAcited by 36
  11. 2023Clinfo.ai: An Open-Source Retrieval-Augmented Large Language Model System for Answering Medical Questions using Scientific Literaturecited by 29
  12. 2023Soft Prompt Tuning for Augmenting Dense Retrieval with Large Language Modelscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 10.

11 papers.

  1. 2025RIPRAG: Hack a Black-box Retrieval-Augmented Generation Question-Answering System with Reinforcement Learningcited by 5
  2. 2026ProGRank: Probe-Gradient Reranking to Defend Dense-Retriever RAG from Corpus Poisoningcited by 2
  3. 2026Distributed Denial of Science: How Indirect Data Poisoning of AI Systems Can Industrialize Scientific Fraudcited by 2
  4. 2026RAGuard: A Layered Defense Framework for Retrieval-Augmented Generation Systems Against Data Poisoningcited by 2
  5. 2026Oracle Poisoning: Corrupting Knowledge Graphs to Weaponise AI Agent Reasoningcited by 1
  6. 2026Combating Knowledge Corruption in Agent Systems: A Byzantine-Tolerant Secure Collaborative RAG Frameworkcited by 1
  7. 2026Influence Factors on RAG Poisoningcited by 0
  8. 2026TriShieldRAG: 3 Rings, One Blind Spot in Layered Defenses for Retrieval-Augmented Generationcited by 0
  9. 2026Defending Retrieval-Augmented Intrusion Detection Against Knowledge Poisoning and Prompt Injectioncited by 0
  10. 2026Mechanism Design for Generative Engines: From Exploitation toward Win-Win Outcomescited by 0
  11. 2026RAGSieve: Self-Referenced Local Contrast for Knowledge-Poison Detection in Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Practical Poisoning Attacks against Retrieval-Augmented Generation
  2. 2025RAG Safety: Exploring Knowledge Poisoning Attacks to Retrieval-Augmented Generation
  3. 2025MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attacks
  4. 2024On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
  5. 2024Towards More Robust Retrieval-Augmented Generation: Evaluating RAG Under Adversarial Poisoning Attacks
  6. 2026RAG-NAROK: Retrieval-Aware Knowledge Corpus Poisoning in RAG with Source-specific Refutation
  7. 2025CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Models
  8. 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models