mlsanityData to 7 Oct 2026

Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agents

arXiv 2411.09523 ↗First posted 14 Nov 2024cs.AIcited by 63

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

With the continuous development of large language models (LLMs), transformer-based models have made groundbreaking advances in numerous natural language processing (NLP) tasks, leading to the emergence of a series of agents that use LLMs as their control hub. While LLMs have achieved success in various tasks, they face numerous security and privacy threats, which become even more severe in the agent scenarios. To enhance the reliability of LLM-based applications, a range of research has emerged to assess and mitigate these risks from different perspectives. To help researchers gain a comprehensive understanding of various risks, this survey collects and analyzes the different threats faced by these agents. To address the challenges posed by previous taxonomies in handling cross-module and cross-stage threats, we propose a novel taxonomy framework based on the sources and impacts. Additionally, we identify six key features of LLM-based agents, based on which we summarize the current research progress and analyze their limitations. Subsequently, we select four representative agents as case studies to analyze the risks they may face in practical use. Finally, based on the aforementioned analyses, we propose future research directions from the perspectives of data, methodology, and policy, respectively.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 172 of its 338 references that are in the map. First posted on arXiv 14 Nov 2024.

Builds on

172 of 338 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
  2. 2024Compromising Embodied Agents with Contextual Backdoor Attacks6 Aug
  3. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
  4. 2024Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-based Decision-Making Systems27 May
  5. 2024AirGapAgent: Protecting Privacy-Conscious Conversational Agents8 May
  6. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
  7. 2024Watch Out for Your Agents! Investigating Backdoor Threats to LLM-Based Agents17 Feb
  8. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
  9. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training10 Jan
  10. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep

Other references · 162

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  9. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  11. 2015Continuous control with deep reinforcement learningcited by 3,279
  12. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
38 more
  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  3. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  5. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  6. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  7. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
  8. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  9. 2023Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Modelscited by 1,047
  10. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  11. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  12. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
  13. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  14. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  15. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  16. 2018Object Hallucination in Image Captioningcited by 661
  17. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  18. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  19. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  20. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  21. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  22. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  23. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  24. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  25. 2023Scalable Extraction of Training Data from (Production) Language Modelscited by 415
  26. 2023Detecting Language Model Attacks with Perplexitycited by 411
  27. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  28. 2023Detecting Pretraining Data from Large Language Modelscited by 375
  29. 2023FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Promptscited by 368
  30. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  31. 2023Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedbackcited by 351
  32. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  33. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  34. 2023Trusting Your Evidence: Hallucinate Less with Context-aware Decodingcited by 319
  35. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 314
  36. 2023How Language Model Hallucinations Can Snowballcited by 310
  37. 2022A Survey of Machine Unlearningcited by 298
  38. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluationcited by 293

50 most cited of 162 All 162 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 33; 2026: 28.

63 papers.

Sort:

By area:

  1. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
  2. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  3. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  4. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  5. 2025Agentic Large Language Models, a surveycited by 105
  6. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
  7. 2025LLM Agents for Education: Advances and Applicationscited by 59
  8. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  9. 2025LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insightscited by 44
  10. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 42
  11. 2025BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attackscited by 37
  12. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
38 more
  1. 2025Towards Trustworthy GUI Agents: A Surveycited by 33
  2. 2025Security Concerns for Large Language Models: A Surveycited by 26
  3. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?cited by 23
  4. 2025Security of Internet of Agents: Attacks and Countermeasurescited by 17
  5. 2025Automatic Red Teaming LLM-based Agents with Model Context Protocol Toolscited by 17
  6. 2025A Survey on Agentic Security: Applications, Threats and Defensescited by 17
  7. 2025Position: Privacy Is Not Just Memorization!cited by 15
  8. 2026MCP-ITP: An Automated Framework for Implicit Tool Poisoning in MCPcited by 13
  9. 2025From Glue-Code to Protocols: A Critical Analysis of A2A and MCP Integration for Scalable Agent Systemscited by 12
  10. 2025Seven Security Challenges in Cross-domain Multi-agent LLM Systemscited by 12
  11. 2025KV Cache Transform Coding for Compact Storage in LLM Inferencecited by 11
  12. 2025Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigationcited by 11
  13. 2026Agentic Artificial Intelligence (AI): Architectures, Taxonomies, and Evaluation of Large Language Model Agentscited by 11
  14. 2026Towards Secure Agent Skills: Architecture, Threat Taxonomy, and Security Analysiscited by 11
  15. 2024On the Generalization and Adaptation Ability of Machine-Generated Text Detectors in Academic Writingcited by 10
  16. 2025Agent Safety Alignment via Reinforcement Learningcited by 10
  17. 2025LLLMs: A Data-Driven Survey of Evolving Research on Limitations of Large Language Modelscited by 9
  18. 2026Your LLM Agent Can Leak Your Data: Data Exfiltration via Backdoored Tool Usecited by 9
  19. 2025LLM Applications: Current Paradigms and the Next Frontiercited by 8
  20. 2026How are AI agents used? Evidence from 177,000 MCP toolscited by 7
  21. 2025Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstructioncited by 6
  22. 2025Toward a Safe Internet of Agentscited by 6
  23. 2026AgentRaft: Automated Detection of Data Over-Exposure in LLM Agentscited by 6
  24. 2026Long Context, Less Focus: A Scaling Gap in LLMs Revealed through Privacy and Personalizationcited by 5
  25. 2026CIBER: A Comprehensive Benchmark for Security Evaluation of Code Interpreter Agentscited by 5
  26. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 5
  27. 2026SafetyDrift: Predicting When AI Agents Cross the Line Before They Actually Docited by 4
  28. 2026Architectural Design Decisions in AI Agent Harnessescited by 4
  29. 2025SoK: Semantic Privacy in Large Language Modelscited by 3
  30. 2026Clawed and Dangerous: Can We Trust Open Agentic Systems?cited by 3
  31. 2026Architecture Matters for Multi-Agent Securitycited by 3
  32. 2026Large Language Models Hack Rewards, and Societycited by 3
  33. 2025Model Editing as a Double-Edged Sword: Steering Agent Ethical Behavior Toward Beneficence or Harmcited by 2
  34. 2025Galaxy: A Cognition-Centered Framework for Proactive, Privacy-Preserving, and Self-Evolving LLM Agentscited by 2
  35. 2025AdaptiveGuard: Towards Adaptive Runtime Safety for LLM-Powered Softwarecited by 2
  36. 2025A Longitudinal Measurement of Privacy Policy Evolution for Large Language Modelscited by 2
  37. 2026AdaptFuse: Training-Free Sequential Preference Learning via Externalized Bayesian Inferencecited by 2
  38. 2026RUBAS: Rubric-Based Reinforcement Learning for Agent Safetycited by 2

50 most cited of 63 All 63 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studies
  2. 2024Unique Security and Privacy Threats of Large Language Models: A Comprehensive Survey
  3. 2024Large Language Model Safety: A Holistic Survey
  4. 2025Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks
  5. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly
  6. 2025A Survey of Attacks on Large Language Models
  7. 2024Current state of LLM Risks and AI Guardrails
  8. 2025SoK: The Privacy Paradox of Large Language Models: Advancements, Privacy Risks, and Mitigation