mlsanityData to 7 Oct 2026

PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety

arXiv 2401.11880 ↗First posted 22 Jan 2024cs.CL cs.AI cs.CRcited by 79

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

Multi-agent systems, when enhanced with Large Language Models (LLMs), exhibit profound capabilities in collective intelligence. However, the potential misuse of this intelligence for malicious purposes presents significant risks. To date, comprehensive research on the safety issues associated with multi-agent systems remains limited. In this paper, we explore these concerns through the innovative lens of agent psychology, revealing that the dark psychological states of agents constitute a significant threat to safety. To tackle these concerns, we propose a comprehensive framework (PsySafe) grounded in agent psychology, focusing on three key areas: firstly, identifying how dark personality traits in agents can lead to risky behaviors; secondly, evaluating the safety of multi-agent systems from the psychological and behavioral perspectives, and thirdly, devising effective strategies to mitigate these risks. Our experiments reveal several intriguing phenomena, such as the collective dangerous behaviors among agents, agents' self-reflection when engaging in dangerous behavior, and the correlation between agents' psychological assessments and dangerous behaviors. We anticipate that our framework and observations will provide valuable insights for further research into the safety of multi-agent systems. We will make our data and code publicly accessible at https://github.com/AI4Good24/PsySafe.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 30 of its 55 references that are in the map. First posted on arXiv 22 Jan 2024.

Builds on

30 of 55 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Evil Geniuses: Delving into the Safety of LLM-based Agents20 Nov

Other references · 29

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  3. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  5. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  6. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  7. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  8. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  9. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 520
  10. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  11. 2023Jailbreaking ChatGPT via Prompt Engineering: An Empirical Studycited by 478
  12. 2021MetaICL: Learning to Learn In Contextcited by 450
17 more
  1. 2023SmoothLLM: Defending Large Language Models Against Jailbreaking Attackscited by 434
  2. 2023Multi-Agent Collaboration: Harnessing the Power of Intelligent LLM Agentscited by 329
  3. 2023Auto-GPT for Online Decision Making: Benchmarks and Additional Opinionscited by 250
  4. 2023Certifying LLM Safety against Adversarial Promptingcited by 248
  5. 2023Defending Against Alignment-Breaking Attacks via Robustly Aligned LLMcited by 173
  6. 2023Image Hijacks: Adversarial Images can Control Generative Models at Runtimecited by 165
  7. 2023Large Language Models Understand and Can be Enhanced by Emotional Stimulicited by 143
  8. 2023Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinkingcited by 78
  9. 2023MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perceptioncited by 67
  10. 2023Emotionally Numb or Empathetic? Evaluating How LLMs Feel Using EmotionBenchcited by 54
  11. 2023Mind meets machine: Unravelling GPT-4's cognitive psychologycited by 31
  12. 2023Who is ChatGPT? Benchmarking LLMs' Psychological Portrayal Using PsychoBenchcited by 28
  13. 2023An Appraisal-Based Chain-Of-Emotion Architecture for Affective Language Model Game Agentscited by 15
  14. 2023ChEF: A Comprehensive Evaluation Framework for Standardized Assessment of Multimodal Large Language Modelscited by 9
  15. 2023The Cultural Psychology of Large Language Models: Is ChatGPT a Holistic or Analytic Thinker?cited by 8
  16. 2023Do Large GPT Models Discover Moral Dimensions in Language Representations? A Topological Study Of Sentence Embeddingscited by 3
  17. 2023ConceptPsy:A Benchmark Suite with Conceptual Comprehensiveness in Psychologycited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 43; 2026: 20.

79 papers.

Sort:

By area:

  1. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 424
  2. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  3. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  4. 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systemscited by 126
  5. 2025Red-Teaming LLM Multi-Agent Systems via Communication Attackscited by 118
  6. 2025A Survey on Trustworthy LLM Agents: Threats and Countermeasurescited by 93
  7. 2024On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agentscited by 92
  8. 2024SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Modelcited by 82
  9. 2024LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shiftscited by 80
  10. 2024CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completioncited by 70
  11. 2024Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Trainingcited by 67
  12. 2025Safety at Scale: A Comprehensive Survey of Large Model and Agent Safetycited by 63
38 more
  1. 2025Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Modelscited by 61
  2. 2025LLM Agents for Education: Advances and Applicationscited by 59
  3. 2025G-Safeguard: A Topology-Guided Security Lens and Treatment on LLM-based Multi-agent Systemscited by 56
  4. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 54
  5. 2024MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Controlcited by 52
  6. 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 47
  7. 2025AgentSafe: Safeguarding Large Language Model-based Multi-agent Systems via Hierarchical Data Managementcited by 35
  8. 2024JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Modelscited by 28
  9. 2024"Not Aligned" is Not "Malicious": Being Careful about Hallucinations of Large Language Models' Jailbreakcited by 23
  10. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?cited by 23
  11. 2024A Trembling House of Cards? Mapping Adversarial Attacks against Language Agentscited by 22
  12. 2025Aegis: Automated Error Generation and Attribution for Multi-Agent Systemscited by 19
  13. 2024Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Modelscited by 18
  14. 2025QueryAttack: Jailbreaking Aligned Large Language Models Using Structured Non-natural Query Languagecited by 16
  15. 2025BlockA2A: Towards Secure and Verifiable Agent-to-Agent Interoperabilitycited by 14
  16. 2025AgentBreeder: Mitigating the AI Safety Risks of Multi-Agent Scaffolds via Self-Improvementcited by 13
  17. 2025Who's the Mole? Modeling and Detecting Intention-Hiding Malicious Agents in LLM-Based Multi-Agent Systemscited by 12
  18. 2025Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Modelscited by 11
  19. 2025IntentionESC: An Intention-Centered Framework for Enhancing Emotional Support in Dialogue Systemscited by 11
  20. 2025From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystemcited by 11
  21. 2025Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Modelscited by 10
  22. 2024AgentMonitor: A Plug-and-Play Framework for Predictive and Secure Multi-Agent Systemscited by 9
  23. 2025When Autonomy Goes Rogue: Preparing for Risks of Multi-Agent Collusion in Social Systemscited by 9
  24. 2025Rethinking Jailbreak Detection of Large Vision Language Models with Representational Contrastive Scoringcited by 9
  25. 2025Amplified Vulnerabilities: Structured Jailbreak Attacks on LLM-based Multi-Agent Debatecited by 8
  26. 2025IP Leakage Attacks Targeting LLM-Based Multi-Agent Systemscited by 8
  27. 2025MedSentry: Understanding and Mitigating Safety Risks in Medical LLM Multi-Agent Systemscited by 8
  28. 2026Risky-Bench: Probing Agentic Safety Risks under Real-World Deploymentcited by 8
  29. 2024Defending LVLMs Against Vision Attacks through Partial-Perception Supervisioncited by 7
  30. 2025Activation Approximations Can Incur Safety Vulnerabilities Even in Aligned LLMs: Comprehensive Analysis and Defensecited by 7
  31. 2025Demonstrations of Integrity Attacks in Multi-Agent Systemscited by 7
  32. 2025SentinelNet: Safeguarding Multi-Agent Collaboration Through Credit-Based Dynamic Threat Detectioncited by 7
  33. 2025When AI Agents Collude Online: Financial Fraud Risks by Collaborative LLM Agents on Social Platformscited by 7
  34. 2025MASTER: Multi-Agent Security Through Exploration of Roles and Topological Structures -- A Comprehensive Frameworkcited by 6
  35. 2025To trust or not to trust: Attention-based Trust Management for LLM Multi-Agent Systemscited by 6
  36. 2025Achilles Heel of Distributed Multi-Agent Systemscited by 5
  37. 2025Can an Individual Manipulate the Collective Decisions of Multi-Agents?cited by 5
  38. 2025PEAR: Planner-Executor Agent Robustness Benchmarkcited by 5

50 most cited of 79 All 79 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Exposing Weak Links in Multi-Agent Systems under Adversarial Prompting
  2. 2025DialogGuard: Multi-Agent Psychosocial Safety Evaluation of Sensitive LLM Responses
  3. 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety
  4. 2026Understanding Cognition-Induced Risks in Agentic AI Systems
  5. 2026"Are You Sure?": An Empirical Study of Human Perception Vulnerability in LLM-Driven Agentic Systems
  6. 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systems
  7. 2025How Brittle is Agent Safety? Rethinking Agent Risk under Intent Concealment and Task Complexity
  8. 2026OpenSkillRisk: Benchmarking Agent Safety When Using Real-World Risky Third-Party Skills