mlsanityData to 14 Oct 2026

Typos that Broke the RAG's Back: Genetic Attack on RAG Pipeline by Simulating Documents in the Wild via Low-level Perturbations

arXiv 2404.13948 ↗First posted 22 Apr 2024cs.CLcited by 65

On the map: Retrieval-augmented generation → Privacy and security of RAG

Abstract (arXiv)

The robustness of recent Large Language Models (LLMs) has become increasingly crucial as their applicability expands across various domains and real-world applications. Retrieval-Augmented Generation (RAG) is a promising solution for addressing the limitations of LLMs, yet existing studies on the robustness of RAG often overlook the interconnected relationships between RAG components or the potential threats prevalent in real-world databases, such as minor textual errors. In this work, we investigate two underexplored aspects when assessing the robustness of RAG: 1) vulnerability to noisy documents through low-level perturbations and 2) a holistic evaluation of RAG robustness. Furthermore, we introduce a novel attack method, the Genetic Attack on RAG (textitGARAG), which targets these aspects. Specifically, GARAG is designed to reveal vulnerabilities within each component and test the overall system functionality against noisy documents. We validate RAG robustness by applying our textitGARAG to standard QA datasets, incorporating diverse retrievers and LLMs. The experimental results show that GARAG consistently achieves high attack success rates. Also, it significantly devastates the performance of each component and their synergy, highlighting the substantial risk that minor textual inaccuracies pose in disrupting RAG systems in the real world.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 32 of its 55 references that are in the map. First posted on arXiv 22 Apr 2024.

Builds on

32 of 55 references.

References · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  7. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
  8. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  9. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,216
  10. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  11. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  12. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
20 more
  1. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  2. 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
  3. 2018Generating Natural Language Adversarial Examplescited by 424
  4. 2023Making Retrieval-Augmented Language Models Robust to Irrelevant Contextcited by 369
  5. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 345
  6. 2024The Power of Noise: Redefining Retrieval for RAG Systemscited by 281
  7. 2022RealTime QA: What's the Answer Right Now?cited by 244
  8. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 232
  9. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 159
  10. 2016Crafting Adversarial Input Sequences for Recurrent Neural Networkscited by 103
  11. 2019Text Processing Like Humans Do: Visually Attacking and Shielding NLP Systemscited by 101
  12. 2021Towards Improving Adversarial Training of NLP Modelscited by 79
  13. 2021The Web Is Your Oyster - Knowledge-Intensive NLP against a Very Large Web Corpuscited by 60
  14. 2020From Hero to Z'eroe: A Benchmark of Low-Level Adversarial Attackscited by 36
  15. 2018WikiAtomicEdits: A Multilingual Corpus of Wikipedia Edits for Modeling Language and Discoursecited by 26
  16. 2023A Survey on Out-of-Distribution Evaluation of Neural NLP Modelscited by 20
  17. 2023How Well Do Large Language Models Truly Ground?cited by 14
  18. 2024Backdoor Attacks on Dense Retrieval via Public and Unintentional Triggerscited by 14
  19. 2022GRS: Combining Generation and Revision in Unsupervised Sentence Simplificationcited by 6
  20. 2023CRYPTEXT: Database and Interactive Toolkit of Human-Written Text Perturbations in the Wildcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 32; 2026: 23.

65 papers.

Sort:

By area:

  1. 2024Certifiably Robust RAG against Retrieval Corruptioncited by 151
  2. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 96
  3. 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Modelscited by 86
  4. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  5. 2024"Glue pizza and eat rocks" -- Exploiting Vulnerabilities in Retrieval-Augmented Generative Modelscited by 60
  6. 2025Practical Poisoning Attacks against Retrieval-Augmented Generationcited by 36
  7. 2024RevPRAG: Revealing Poisoning Attacks in Retrieval-Augmented Generation through LLM Activation Analysiscited by 33
  8. 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Modelscited by 33
  9. 2025RbFT: Robust Fine-tuning for Retrieval-Augmented Generation against Retrieval Defectscited by 27
  10. 2024OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generationcited by 26
  11. 2025FlippedRAG: Black-Box Opinion Manipulation Adversarial Attacks to Retrieval-Augmented Generation Modelscited by 25
  12. 2025MM-PoisonRAG: Disrupting Multimodal RAG with Local and Global Poisoning Attackscited by 24
38 more
  1. 2025RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Modelscited by 24
  2. 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 18
  3. 2025ReliabilityRAG: Effective and Provably Robust Defense for RAG-based Web-Searchcited by 18
  4. 2025CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Modelscited by 17
  5. 2024Adversarial Decoding: Generating Readable Documents for Adversarial Objectivescited by 16
  6. 2025The RAG Paradox: A Black-Box Attack Exploiting Unintentional Vulnerabilities in Retrieval-Augmented Generation Systemscited by 14
  7. 2025Poisoned-MRAG: Knowledge Poisoning Attacks to Multimodal Retrieval Augmented Generationcited by 12
  8. 2024Black-Box Opinion Manipulation Attacks to Retrieval-Augmented Generation of Large Language Modelscited by 11
  9. 2025Out of Style: RAG's Fragility to Linguistic Variationcited by 11
  10. 2025Defending Against Knowledge Poisoning Attacks During Retrieval-Augmented Generationcited by 11
  11. 2025Quantifying and Improving the Robustness of Retrieval-Augmented Language Models Against Spurious Features in Grounding Datacited by 10
  12. 2025Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attackcited by 10
  13. 2025The Silent Saboteur: Imperceptible Adversarial Attacks against Black-Box Retrieval-Augmented Generation Systemscited by 10
  14. 2024A Reality Check on Context Utilisation for Retrieval-Augmented Generationcited by 9
  15. 2025Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understandingcited by 9
  16. 2025Rescuing the Unpoisoned: Efficient Defense against Knowledge Corruption Attacks on RAG Systemscited by 9
  17. 2025Your RAG is Unfair: Exposing Fairness Vulnerabilities in Retrieval-Augmented Generation via Backdoor Attackscited by 8
  18. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systemscited by 7
  19. 2025Redefining Machine Translation on Social Network Services with Large Language Modelscited by 6
  20. 2025Chain-of-Thought Poisoning Attacks against R1-based Retrieval-Augmented Generation Systemscited by 6
  21. 2025Evaluating the Robustness of Retrieval-Augmented Generation to Adversarial Evidence in the Health Domaincited by 6
  22. 2025Parametric Retrieval-Augmented Generation using Latent Routing of LoRA Adapterscited by 6
  23. 2025Uncovering Competing Poisoning Attacks in Retrieval-Augmented Generationcited by 5
  24. 2025Token-Level Precise Attack on RAG: Searching for the Best Alternatives to Mislead Generationcited by 5
  25. 2025AIP: Subverting Retrieval-Augmented Generation via Adversarial Instructional Promptcited by 5
  26. 2025MIRAGE: Misleading Retrieval-Augmented Generation via Black-box and Query-agnostic Poisoning Attackscited by 4
  27. 2025UniC-RAG: Universal Knowledge Corruption Attacks to Retrieval-Augmented Generationcited by 3
  28. 2026Connect the Dots: Knowledge Graph-Guided Crawler Attack on Retrieval-Augmented Generation Systemscited by 3
  29. 2025Fact or Facsimile? Evaluating the Factual Robustness of Modern Retrieverscited by 2
  30. 2026Making Theft Useless: Adulteration-Based Protection of Proprietary Knowledge Graphs in GraphRAG Systemscited by 2
  31. 2026ProGRank: Probe-Gradient Reranking to Defend Dense-Retriever RAG from Corpus Poisoningcited by 2
  32. 2026Securing Retrieval-Augmented Generation: A Taxonomy of Attacks, Defenses, and Future Directionscited by 2
  33. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1
  34. 2026Reliability-Aware Determinantal Point Processes for Robust Informative Data Selection in Large Language Modelscited by 1
  35. 2026Needle-in-RAG: Prompt-Conditioned Character-Level Traceback of Poisoned Spans in Retrieved Evidencecited by 1
  36. 2026RADAR: Defending RAG Dynamically against Retrieval Corruptioncited by 1
  37. 2026DiscourseFlip: An Oblique Discourse-Level Opinion Manipulation Attack against Black-box Retrieval-Augmented Generationcited by 1
  38. 2026Conflict-Aware Retriever Editing for Knowledge Injection Attacks on LLM-Based RAG Systemscited by 1

50 most cited of 65 All 65 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025RAGPart & RAGMask: Retrieval-Stage Defenses Against Corpus Poisoning in Retrieval-Augmented Generation
  2. 2026PRA-RAG: Provably Robust Aggregation in Retrieval-Augmented Generation against Retrieval Corruption
  3. 2025RAG Security and Privacy: Formalizing the Threat Model and Attack Surface
  4. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
  5. 2025Evaluating the Retrieval Robustness of Large Language Models
  6. 2025PR-Attack: Coordinated Prompt-RAG Attacks on Retrieval-Augmented Generation in Large Language Models via Bilevel Optimization
  7. 2024On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains
  8. 2024C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models