mlsanityData to 13 Oct 2026

AgentSafe: Safeguarding Large Language Model-based Multi-agent Systems via Hierarchical Data Management

arXiv 2503.04392 ↗First posted 6 Mar 2025cs.AIcited by 36

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Large Language Model based multi-agent systems are revolutionizing autonomous communication and collaboration, yet they remain vulnerable to security threats like unauthorized access and data breaches. To address this, we introduce AgentSafe, a novel framework that enhances MAS security through hierarchical information management and memory protection. AgentSafe classifies information by security levels, restricting sensitive data access to authorized agents. AgentSafe incorporates two components: ThreatSieve, which secures communication by verifying information authority and preventing impersonation, and HierarCache, an adaptive memory management system that defends against unauthorized access and malicious poisoning, representing the first systematic defense for agent memory. Experiments across various LLMs show that AgentSafe significantly boosts system resilience, achieving defense success rates above 80% under adversarial conditions. Additionally, AgentSafe demonstrates scalability, maintaining robust performance as agent numbers and information complexity grow. Results underscore effectiveness of AgentSafe in securing MAS and its potential for real-world application.

Key facts. Cited by 36 papers in the map (data through 13 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 25 of its 42 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

25 of 42 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases17 Jul
  2. 2024The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operative20 Feb
  3. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast13 Feb
  4. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety22 Jan

Other references · 21

  1. 2023GPT-4 Technical Reportcited by 18,278
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,117
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,218
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,777
  5. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,758
  6. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,405
  7. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,404
  8. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,365
  9. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,001
  10. 2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 565
  11. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 556
  12. 2023DeepInception: Hypnotize Large Language Model to Be Jailbreakercited by 361
9 more
  1. 2023Multi-step Jailbreaking Privacy Attacks on ChatGPTcited by 333
  2. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 331
  3. 2023A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easilycited by 259
  4. 2023Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulationcited by 211
  5. 2023OpenAgents: An Open Platform for Language Agents in the Wildcited by 136
  6. 2024Attacks, Defenses and Evaluations for LLM Conversation Safety: A Surveycited by 130
  7. 2024Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applicationscited by 59
  8. 2024Large Language Model for Participatory Urban Planningcited by 40
  9. 2024AI-Driven Virtual Teacher for Enhanced Educational Efficiency: Leveraging Large Pretrain Models for Autonomous Error Analysis and Correctioncited by 12

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 19; 2026: 17.

36 papers.

Sort:

By area:

  1. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  2. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 114
  3. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 94
  4. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 55
  5. 2026Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Frameworkcited by 52
  6. 2025We Should Identify and Mitigate Third-Party Safety Risks in MCP-Powered Agent Systemscited by 27
  7. 2025Attack the Messages, Not the Agents: A Multi-round Adaptive Stealthy Tampering Framework for LLM-MAScited by 17
  8. 2025SafeScientist: Toward Risk-Aware Scientific Discoveries by LLM Agentscited by 15
  9. 2025Graph-Augmented Large Language Model Agents: Current Progress and Future Prospectscited by 14
  10. 2025SecureVibeBench: Benchmarking Secure Vibe Coding of AI Agents via Reconstructing Vulnerability-Introducing Scenarioscited by 12
  11. 2025Benchmarking the Robustness of Agentic Systems to Adversarially-Induced Harmscited by 10
  12. 2025Explainable and Fine-Grained Safeguarding of LLM Multi-Agent Systems via Bi-Level Graph Anomaly Detectioncited by 8
24 more
  1. 2026Risky-Bench: Probing Agentic Safety Risks under Real-World Deploymentcited by 8
  2. 2025Goal-Aware Identification and Rectification of Misinformation in Multi-Agent Systemscited by 7
  3. 2025Deep Research with Open-Domain Evaluation and Multi-Stage Guardrails for Safetycited by 7
  4. 2025SentinelNet: Safeguarding Multi-Agent Collaboration Through Credit-Based Dynamic Threat Detectioncited by 7
  5. 2025MASTER: Multi-Agent Security Through Exploration of Roles and Topological Structures -- A Comprehensive Frameworkcited by 6
  6. 2026INFA-Guard: Mitigating Malicious Propagation via Infection-Aware Safeguarding in LLM-Based Multi-Agent Systemscited by 5
  7. 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 5
  8. 2025Don't Trust Your Upstream: Exploiting LLM Multi-Agent System via Topology-Guided Adversarial Propagationcited by 3
  9. 2025GAMA: A General Anonymizing Multi-Agent System for Privacy Preservation Enhanced by Domain Rules and Disproof Mechanismcited by 2
  10. 2026AgenticCyOps: Securing Multi-Agentic AI Integration in Enterprise Cyber Operationscited by 2
  11. 2026DynaTrust: Defending Multi-Agent Systems Against Sleeper Agents via Dynamic Trust Graphscited by 2
  12. 2026The Capability Paradox: How Smarter Auditors Make Multi-Agent Systems Less Securecited by 2
  13. 2025From static to adaptive: immune memory-based jailbreak detection for large language modelscited by 1
  14. 2026Spider-Sense: Intrinsic Risk Sensing for Efficient Agent Defense with Hierarchical Adaptive Screeningcited by 1
  15. 2026Granulon: Awakening Pixel-Level Visual Encoders with Adaptive Multi-Granularity Semantics for MLLMcited by 1
  16. 2026SciTrace: Trajectory-Aware Safety Reasoning for Scientific Discovery Agentscited by 1
  17. 2026AutoRAS: Learning Robust Agentic Systems with Primitive Representationscited by 1
  18. 2026When Agents Go Rogue: Activation-Based Detection of Malicious Behaviors in Multi-Agent Systemscited by 1
  19. 2026Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directionscited by 1
  20. 2026SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systemscited by 1
  21. 2025Multi-scale Temporal Prediction via Incremental Generation and Multi-agent Collaborationcited by 0
  22. 2026OpenEvoShield: Dual Non-Stationary Continual Defense for Open-World Multi-Agent System Attackscited by 0
  23. 2026MAPLE-Guard: Memory-Aware Link Enforcement Against Memory-Link Poisoning in Multi-Agent Systemscited by 0
  24. 2026When the Agent Becomes the Kernel: A Systematization of Security on the Path to AI-Native Operating Systemscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024AgentMonitor: A Plug-and-Play Framework for Predictive and Secure Multi-Agent Systems
  2. 2025A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory
  3. 2026From Storage to Steering: Memory Control Flow Attacks on LLM Agents
  4. 2026Architecting Secure AI Agents: Perspectives on System-Level Defenses Against Indirect Prompt Injection Attacks
  5. 2026When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems
  6. 2025Unveiling Privacy Risks in LLM Agent Memory
  7. 2025DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents
  8. 2025AgentSentinel: An End-to-End and Real-Time Security Defense Framework for Computer-Use Agents