mlsanityData to 7 Oct 2026

TrojanRAG: Retrieval-Augmented Generation Can Be Backdoor Driver in Large Language Models

arXiv 2405.13401 ↗First posted 22 May 2024cs.CR cs.CLcited by 63

On the map: Retrieval-augmented generation → Privacy and security of RAG

also in 1 more
Abstract (arXiv)

Large language models (LLMs) have raised concerns about potential security threats despite performing significantly in Natural Language Processing (NLP). Backdoor attacks initially verified that LLM is doing substantial harm at all stages, but the cost and robustness have been criticized. Attacking LLMs is inherently risky in security review, while prohibitively expensive. Besides, the continuous iteration of LLMs will degrade the robustness of backdoors. In this paper, we propose TrojanRAG, which employs a joint backdoor attack in the Retrieval-Augmented Generation, thereby manipulating LLMs in universal attack scenarios. Specifically, the adversary constructs elaborate target contexts and trigger sets. Multiple pairs of backdoor shortcuts are orthogonally optimized by contrastive learning, thus constraining the triggering conditions to a parameter subspace to improve the matching. To improve the recall of the RAG for the target contexts, we introduce a knowledge graph to construct structured data to achieve hard matching at a fine-grained level. Moreover, we normalize the backdoor scenarios in LLMs to analyze the real harm caused by backdoors from both attackers' and users' perspectives and further verify whether the context is a favorable tool for jailbreaking models. Extensive experimental results on truthfulness, language understanding, and harmfulness show that TrojanRAG exhibits versatility threats while maintaining retrieval capabilities on normal queries.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Placed in Privacy and security of RAG, one of 15 directions in Retrieval-augmented generation. Builds on 41 of its 60 references that are in the map. First posted on arXiv 22 May 2024.

Builds on

41 of 60 references.

References · 41

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  6. 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
  7. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  8. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  9. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  10. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
  11. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  12. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
29 more
  1. 2020Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrievalcited by 846
  2. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 705
  3. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  4. 2019Passage Re-ranking with BERTcited by 629
  5. 2022MTEB: Massive Text Embedding Benchmarkcited by 623
  6. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  7. 2019Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoringcited by 164
  8. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injectioncited by 143
  9. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agentscited by 135
  10. 2020UnQovering Stereotyping Biases via Underspecified Questionscited by 124
  11. 2023Universal Jailbreak Backdoors from Poisoned Human Feedbackcited by 120
  12. 2021Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoningcited by 103
  13. 2023Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documentscited by 103
  14. 2023AnglE-optimized Text Embeddingscited by 98
  15. 2023Backdoor Attacks for In-Context Learning with Language Modelscited by 92
  16. 2023Prompt as Triggers for Backdoor Attack: Examining the Vulnerability in Language Modelscited by 87
  17. 2024BadEdit: Backdooring large language models by model editingcited by 84
  18. 2020Relevance-guided Supervision for OpenQA with ColBERTcited by 71
  19. 2023BadGPT: Exploring Security Vulnerabilities of ChatGPT via Backdoor Attacks to InstructGPTcited by 70
  20. 2023Composite Backdoor Attacks Against Large Language Modelscited by 63
  21. 2024Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledgecited by 48
  22. 2024Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learningcited by 47
  23. 2023PoisonPrompt: Backdoor Attack on Prompt-based Large Language Modelscited by 40
  24. 2023Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injectionscited by 37
  25. 2023Backdoor Attacks and Countermeasures in Natural Language Processing Models: A Comprehensive Security Reviewcited by 34
  26. 2023TrojLLM: A Black-box Trojan Prompt Attack on Large Language Modelscited by 29
  27. 2023Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignmentcited by 27
  28. 2023Poisoned ChatGPT Finds Work for Idle Hands: Exploring Developers' Coding Practices with Insecure Suggestions from Poisoned AI Modelscited by 13
  29. 2024Backdoor Attacks on Dense Retrieval via Public and Unintentional Triggerscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 31; 2026: 22.

63 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2024Trustworthiness in Retrieval-Augmented Generation Systems: A Surveycited by 83
  3. 2025Practical Poisoning Attacks against Retrieval-Augmented Generationcited by 35
  4. 2024Jailbreaking and Mitigation of Vulnerabilities in Large Language Modelscited by 33
  5. 2025Topic-FlipRAG: Topic-Orientated Adversarial Opinion Manipulation Attacks to Retrieval-Augmented Generation Modelscited by 32
  6. 2024PoisonBench: Assessing Large Language Model Vulnerability to Data Poisoningcited by 31
  7. 2024Unleashing Worms and Extracting Data: Escalating the Outcome of Attacks against RAG-based Inference in Scale and Severity Using Jailbreakingcited by 30
  8. 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystemscited by 30
  9. 2025GraphRAG under Firecited by 23
  10. 2024ConfusedPilot: Confused Deputy Risks in RAG-based LLMscited by 22
  11. 2026SkillTrojan: Backdoor Attacks on Skill-Based Agent Systemscited by 21
  12. 2024When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge?cited by 20
38 more
  1. 2025Benchmarking Poisoning Attacks against Retrieval-Augmented Generationcited by 17
  2. 2025Joint-GCG: Unified Gradient-Based Poisoning Attacks on Retrieval-Augmented Generation Systemscited by 16
  3. 2025Unsafe LLM-Based Search: Quantitative Analysis and Mitigation of Safety Risks in AI Web Searchcited by 15
  4. 2024Mitigating Backdoor Threats to Large Language Models: Advancement and Challengescited by 14
  5. 2025Who Taught the Lie? Responsibility Attribution for Poisoned Knowledge in Retrieval-Augmented Generationcited by 12
  6. 2025SeCon-RAG: A Two-Stage Semantic Filtering and Conflict-Free Framework for Trustworthy RAGcited by 12
  7. 2025AI Meets Brain: Memory Systems from Cognitive Neuroscience to Autonomous Agentscited by 12
  8. 2026BackdoorAgent: A Unified Framework for Backdoor Attacks on LLM-based Agentscited by 12
  9. 2025TracLLM: A Generic Framework for Attributing Long Context LLMscited by 11
  10. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  11. 2024The Dark Side of Trust: Authority Citation-Driven Jailbreak Attacks on Large Language Modelscited by 9
  12. 2025Rescuing the Unpoisoned: Efficient Defense against Knowledge Corruption Attacks on RAG Systemscited by 9
  13. 2024Blockchain for Large Language Model Security and Safety: A Holistic Surveycited by 8
  14. 2025One Pic is All it Takes: Poisoning Visual Document Retrieval Augmented Generation with a Single Imagecited by 8
  15. 2025Bias Amplification in RAG: Poisoning Knowledge Retrieval to Steer LLMscited by 8
  16. 2025Position: LLM Watermarking Should Align Stakeholders' Incentives for Practical Adoptioncited by 7
  17. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systemscited by 7
  18. 2025CtrlRAG: Black-box Document Poisoning Attacks for Retrieval-Augmented Generation of Large Language Modelscited by 5
  19. 2025AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruptioncited by 5
  20. 2025Towards Copyright Protection for Knowledge Bases of Retrieval-augmented Language Models via Reasoningcited by 4
  21. 2025The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromisecited by 4
  22. 2025ImageSentinel: Protecting Visual Datasets from Unauthorized Retrieval-Augmented Image Generationcited by 4
  23. 2026BraveGuard: From Open-World Threats to Safer Computer-Use Agentscited by 4
  24. 2024Towards Data Governance of Frontier AI Modelscited by 3
  25. 2025Hoist with His Own Petard: Inducing Guardrails to Facilitate Denial-of-Service Attacks on Retrieval-Augmented Generation of LLMscited by 3
  26. 2025UniC-RAG: Universal Knowledge Corruption Attacks to Retrieval-Augmented Generationcited by 3
  27. 2025RAG Makes Guardrails Unsafe? Investigating Robustness of Guardrails under RAG-style Contextscited by 3
  28. 2026DRAINCODE: Stealthy Energy Consumption Attacks on Retrieval-Augmented Code Generation via Context Poisoningcited by 3
  29. 2026MEMSAD: Gradient-Coupled Anomaly Detection for Memory Poisoning in Retrieval-Augmented Agentscited by 3
  30. 2025DisarmRAG: Stealthy Retriever-Centric Poisoning to Disable Self-Correction in Retrieval-Augmented Generation (Extended Version)cited by 2
  31. 2025Safeguarding Multimodal Knowledge Copyright in the RAG-as-a-Service Environmentcited by 1
  32. 2025LLM in the Middle: A Systematic Review of Threats and Mitigations to Real-World LLM-based Systemscited by 1
  33. 2025Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractorscited by 1
  34. 2026Needle-in-RAG: Prompt-Conditioned Character-Level Traceback of Poisoned Spans in Retrieved Evidencecited by 1
  35. 2026Detecting Is Not Resolving: The Monitoring Control Gap in Retrieval Augmented LLMscited by 1
  36. 2026DiscourseFlip: An Oblique Discourse-Level Opinion Manipulation Attack against Black-box Retrieval-Augmented Generationcited by 1
  37. 2026Conflict-Aware Retriever Editing for Knowledge Injection Attacks on LLM-Based RAG Systemscited by 1
  38. 2025Provably Secure Retrieval-Augmented Generationcited by 0

50 most cited of 63 All 63 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models
  2. 2024Data Extraction Attacks in Retrieval-Augmented Generation via Backdoors
  3. 2024Backdoored Retrievers for Prompt Injection Attacks on Retrieval Augmented Generation of Large Language Models
  4. 2025Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputs
  5. 2024BadRAG: Identifying Vulnerabilities in Retrieval Augmented Generation of Large Language Models
  6. 2026Retrieved But Not Reliable: A Survey on Attacks, and Defenses in Retrieval-Augmented Generation
  7. 2024PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
  8. 2024Phantom: General Backdoor Attacks on Retrieval Augmented Language Generation