mlsanityData to 7 Oct 2026

Towards Unsupervised Adversarial Document Detection in Retrieval Augmented Generation Systems

arXiv 2603.17176 ↗First posted 17 Mar 2026cs.CR cs.AIcited by 0

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

Retrieval augmented generation systems have become an integral part of everyday life. Whether in internet search engines, email systems, or service chatbots, these systems are based on context retrieval and answer generation with large language models. With their spread, also the security vulnerabilities increase. Attackers become increasingly focused on these systems and various hacking approaches are developed. Manipulating the context documents is a way to persist attacks and make them affect all users. Therefore, detecting compromised, adversarial context documents early is crucial for security. While supervised approaches require a large amount of labeled adversarial contexts, we propose an unsupervised approach, being able to detect also zero day attacks. We conduct a preliminary study to show appropriate indicators for adversarial contexts. For that purpose generator activations, output embeddings, and an entropy-based uncertainty measure turn out as suitable, complementary quantities. With an elementary statistical outlier detection, we propose and compare their detection abilities. Furthermore, we show that the target prompt, which the attacker wants to manipulate, is not required for a successful detection. Moreover, our results indicate that a simple context summary generation might even be superior in finding manipulated contexts.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 16 of its 25 references that are in the map. First posted on arXiv 17 Mar 2026.

Builds on

16 of 25 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Safeguarding Privacy of Retrieval Data against Membership Inference Attacks: Is This Query Too Close to Home?28 May
  2. 2024RevPRAG: Revealing Poisoning Attacks in Retrieval-Augmented Generation through LLM Activation Analysis28 Nov
  3. 2024Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents9 Jun
  4. 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models3 Jun
  5. 2024Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations22 Apr
  6. 2024The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)23 Feb
  7. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models12 Feb

Other references · 9

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
  6. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  7. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
  8. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 147
  9. 2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Modelscited by 34

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
  2. 2025The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systems
  3. 2024Towards More Robust Retrieval-Augmented Generation: Evaluating RAG Under Adversarial Poisoning Attacks
  4. 2025GRADA: Graph-based Reranking against Adversarial Documents Attack
  5. 2025TechniqueRAG: Retrieval Augmented Generation for Adversarial Technique Annotation in Cyber Threat Intelligence Text
  6. 2024Toward Robust RALMs: Revealing the Impact of Imperfect Retrieval on Retrieval-Augmented Language Models
  7. 2026SilentRetrieval: Hijacking Retrieval-Augmented Generation via Semantically-Preserving Adversarial Data Poisoning
  8. 2025AIP: Subverting Retrieval-Augmented Generation via Adversarial Instructional Prompt