mlsanityData to 7 Oct 2026

On the Resilience of LLM-Based Multi-Agent Collaboration with Faulty Agents

arXiv 2408.00989 ↗First posted 2 Aug 2024cs.AIcited by 92

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Large language model-based multi-agent systems have shown great abilities across various tasks due to the collaboration of expert agents, each focusing on a specific domain. However, the impact of clumsy or even malicious agents--those who frequently make errors in their tasks--on the overall performance of the system remains underexplored. This paper investigates: (1) What is the resilience of various system structures (e.g., ArightarrowBrightarrowC, AleftrightarrowBleftrightarrowC) under faulty agents, on different downstream tasks? (2) How can we increase system resilience to defend against these agents? To simulate faulty agents, we propose two approaches--AutoTransform and AutoInject--which introduce mistakes into the agents' responses. Experiments on four downstream tasks using six systems show that the "hierarchical" structure, i.e., Arightarrow(BleftrightarrowC), exhibits superior resilience with the lowest performance drop of 5.5%, compared to 10.5% and 23.7% of other two structures. To further improve resilience, we introduce (1) Challenger, that introduces a mechanism for each agent to challenge others' outputs, and (2) Inspector, an additional agent to review and correct messages, recovering up to 96.4% errors made by faulty agents. Our code and data are available at https://github.com/CUHK-ARISE/MAS-Resilience.

Key facts. Cited by 92 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 41 of its 66 references that are in the map. First posted on arXiv 2 Aug 2024.

Builds on

41 of 66 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence9 Jul
  2. 2024Scaling Large Language Model-based Multi-Agent Collaboration11 Jun
  3. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts20 May
  4. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents18 Oct
  5. 2023AutoAgents: A Framework for Automatic Agent Generation29 Sep
  6. 2023Agents: An Open-source Framework for Autonomous Language Agents14 Sep
  7. 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors21 Aug
  8. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate14 Aug
  9. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
  10. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration11 Jul
  11. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society31 Mar

Other references · 30

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  6. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  7. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  8. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,413
  9. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  10. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  11. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  12. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
18 more
  1. 2022Red Teaming Language Models with Language Modelscited by 983
  2. 2023ChatDev: Communicative Agents for Software Developmentcited by 893
  3. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  4. 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 328
  5. 2023Self-collaboration Code Generation via ChatGPTcited by 255
  6. 2024Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agentscited by 210
  7. 2023InCharacter: Evaluating Personality Fidelity in Role-Playing Agents through Psychological Interviewscited by 177
  8. 2024Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fastcited by 149
  9. 2023Evil Geniuses: Delving into the Safety of LLM-based Agentscited by 103
  10. 2021Learning Compact Metrics for MTcited by 93
  11. 2024Flooding Spread of Manipulated Knowledge in LLM-Based Multi-Agent Communitiescited by 84
  12. 2024PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safetycited by 79
  13. 2024Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMscited by 59
  14. 2024MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debatecited by 54
  15. 2024UniDebugger: Hierarchical Multi-Agent Framework for Unified Software Debuggingcited by 35
  16. 2021Reliability Testing for Natural Language Processing Systemscited by 25
  17. 2024InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Contextcited by 25
  18. 2023Leveraging Word Guessing Games to Assess the Intelligence of Large Language Modelscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 36; 2026: 55.

92 papers.

Sort:

By area:

  1. 2024Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systemscited by 126
  2. 2025Advances and Challenges in Foundation Agents: From Brain-Inspired Intelligence to Evolutionary, Collaborative, and Safe Systemscited by 125
  3. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 110
  4. 2025MasRouter: Learning to Route LLMs for Multi-Agent Systemscited by 80
  5. 2025Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agentscited by 66
  6. 2025Multi-Agent Systems Execute Arbitrary Malicious Codecited by 52
  7. 2025GUARDIAN: Safeguarding LLM Multi-Agent Collaborations with Temporal Graph Modelingcited by 35
  8. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Userscited by 32
  9. 2025Peacemaker or Troublemaker: How Sycophancy Shapes Multi-Agent Debatecited by 29
  10. 2025Agentic AI Security: Threats, Defenses, Evaluation, and Open Challengescited by 26
  11. 2025A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?cited by 23
  12. 2025Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewardscited by 23
38 more
  1. 2026Large Language Model Reasoning Failurescited by 20
  2. 2025Aegis: Automated Error Generation and Attribution for Multi-Agent Systemscited by 19
  3. 2025Attack the Messages, Not the Agents: A Multi-round Adaptive Stealthy Tampering Framework for LLM-MAScited by 17
  4. 2025MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQLcited by 14
  5. 2025Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatchcited by 13
  6. 2025The PIMMUR Principles: Ensuring Validity in Collective Behavior of LLM Societiescited by 12
  7. 2025Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberationcited by 11
  8. 2025SoK: Trust-Authorization Mismatch in LLM Agent Interactionscited by 11
  9. 2026MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systemscited by 11
  10. 2026Who&When Pro: Can LLMs Really Attribute Failures in AI Agents?cited by 11
  11. 2025Topology Matters: Measuring Memory Leakage in Multi-Agent LLMscited by 9
  12. 2026DART: Semantic Recoverability for Structured Tool Agentscited by 9
  13. 2025IP Leakage Attacks Targeting LLM-Based Multi-Agent Systemscited by 8
  14. 2025SentinelNet: Safeguarding Multi-Agent Collaboration Through Credit-Based Dynamic Threat Detectioncited by 7
  15. 2025When AI Agents Collude Online: Financial Fraud Risks by Collaborative LLM Agents on Social Platformscited by 7
  16. 2025Shadows in the Code: Exploring the Risks and Defenses of LLM-based Multi-Agent Software Development Systemscited by 7
  17. 2026Thought Virus: Viral Misalignment via Subliminal Prompting in Multi-Agent Systemscited by 7
  18. 2026AutoScientists: Self-Organizing Agent Teams for Long-Running Scientific Experimentationcited by 7
  19. 2026The Value of Variance: Mitigating Debate Collapse in Multi-Agent Systems via Uncertainty-Driven Policy Optimizationcited by 6
  20. 2026AgentFixer: From Failure Detection to Fix Recommendations in LLM Agentic Systemscited by 6
  21. 2026The Chameleon's Limit: Investigating Persona Collapse and Homogenization in Large Language Modelscited by 6
  22. 2025PEAR: Planner-Executor Agent Robustness Benchmarkcited by 5
  23. 2025Multimodal Safety Evaluation in Generative Agent Social Simulationscited by 5
  24. 2025Formalizing the Safety, Security, and Functional Properties of Agentic AI Systemscited by 5
  25. 2026ResMAS: Resilience Optimization in LLM-based Multi-agent Systemscited by 5
  26. 2026INFA-Guard: Mitigating Malicious Propagation via Infection-Aware Safeguarding in LLM-Based Multi-Agent Systemscited by 5
  27. 2026ACIArena: Toward Unified Evaluation for Agent Cascading Injectioncited by 5
  28. 2025Evo-MARL: Co-Evolutionary Multi-Agent Reinforcement Learning for Internalized Safetycited by 4
  29. 2025With Great Capabilities Come Great Responsibilities: Introducing the Agentic Risk & Capability Framework for Governing Agentic AI Systemscited by 4
  30. 2025MASCA: LLM based-Multi Agents System for Credit Assessmentcited by 3
  31. 2025AdvEvo-MARL: Shaping Internalized Safety through Adversarial Co-Evolution in Multi-Agent Reinforcement Learningcited by 3
  32. 2025Securing Multi-Agent Systems Against Corruptions via Node Contribution Backpropagationcited by 3
  33. 2025MAS-Shield: A Defense Framework for Secure and Efficient LLM MAScited by 3
  34. 2025Don't Trust Your Upstream: Exploiting LLM Multi-Agent System via Topology-Guided Adversarial Propagationcited by 3
  35. 2026TextResNet: Decoupling and Routing Optimization Signals in Compound AI Systems via Deep Residual Tuningcited by 3
  36. 2026Architecting Trust in Artificial Epistemic Agentscited by 3
  37. 2026CIA: Inferring the Communication Topology from LLM-based Multi-Agent Systemscited by 3
  38. 2025Analyzing Code Injection Attacks on LLM-based Multi-Agent Systems in Software Developmentcited by 2

50 most cited of 92 All 92 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents
  2. 2026Silent Failure in LLM Agent Systems: The Entropy Principle and the Inevitable Disorder of Autonomous Agents
  3. 2025Where LLM Agents Fail and How They can Learn From Failures
  4. 2026A Self-Healing Framework for Reliable LLM-Based Autonomous Agents
  5. 2025Aegis: Taxonomy and Optimizations for Overcoming Agent-Environment Failures in LLM Agents
  6. 2025LLMs Working in Harmony: A Survey on the Technological Aspects of Building Effective LLM-Based Multi Agent Systems
  7. 2025How Do LLMs Fail In Agentic Scenarios? A Qualitative Analysis of Success and Failure Scenarios of Various LLMs in Agentic Simulations
  8. 2026From Task Solving to Robust Real-World Adaptation in LLM Agents