mlsanityData to 14 Oct 2026

An Outlook on the Opportunities and Challenges of Multi-Agent AI Systems

arXiv 2505.18397 ↗First posted 23 May 2025cs.MA cs.AI cs.ETcited by 8

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

A multi-agent AI system (MAS) is composed of multiple autonomous agents that interact, exchange information, and make decisions based on internal generative models. Recent advances in large language models and tool-using agents have made MAS increasingly practical in areas like scientific discovery and collaborative automation. However, key questions remain: When are MAS more effective than single-agent systems? What new safety risks arise from agent interactions? And how should we evaluate their reliability and structure? This paper outlines a formal framework for analyzing MAS, focusing on two core aspects: effectiveness and safety. We explore whether MAS truly improve robustness, adaptability, and performance, or merely repackage known techniques like ensemble learning. We also study how inter-agent dynamics may amplify or suppress system vulnerabilities. While MAS are relatively new to the signal processing community, we envision them as a powerful abstraction that extends classical tools like distributed estimation and sensor fusion to higher-level, policy-driven inference. Through experiments on data science automation, we highlight the potential of MAS to reshape how signal processing systems are designed and trusted.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 27 of its 69 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

27 of 69 references.

References · 27

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2013Intriguing properties of neural networkscited by 5,155
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  4. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  6. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
  7. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  8. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,055
  9. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
  10. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  11. 2023Prompt Injection attack against LLM-integrated Applicationscited by 605
  12. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
15 more
  1. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
  2. 2020Invisible Backdoor Attack with Sample-Specific Triggerscited by 353
  3. 2017Adversarial Examples: Attacks and Defenses for Deep Learningcited by 351
  4. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  5. 2019Emergent Tool Use From Multi-Agent Autocurriculacited by 313
  6. 2023Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectivescited by 257
  7. 2017Multi-agent Reinforcement Learning in Sequential Social Dilemmascited by 209
  8. 2018A Survey and Critique of Multiagent Deep Reinforcement Learningcited by 177
  9. 2025Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systemscited by 62
  10. 2019Deep Reinforcement Learning from Policy-Dependent Human Feedbackcited by 56
  11. 2024MAP: Multi-Human-Value Alignment Palettecited by 20
  12. 2023Demystifying Poisoning Backdoor Attacks from a Statistical Perspectivecited by 10
  13. 2016Towards Multi-Agent Communication-Based Language Learningcited by 7
  14. 2021Interval Privacy: A Framework for Privacy-Preserving Data Collectioncited by 3
  15. 2025Model Privacy: A Unified Framework for Understanding Model Stealing Attacks and Defensescited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 2.

8 papers.

  1. 2025TRiSM for Agentic AI: A Review of Trust, Risk, and Security Management in LLM-based Agentic Multi-Agent Systemscited by 59
  2. 2025CoDA: Agentic Systems for Collaborative Data Visualizationcited by 15
  3. 2025Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatchcited by 14
  4. 2025Stop Wasting Your Tokens: Towards Efficient Runtime Multi-Agent Systemscited by 12
  5. 2025An LLM-based Agentic Framework for Accessible Network Controlcited by 2
  6. 2026MA-CoNav: A Master-Slave Multi-Agent Framework with Hierarchical Collaboration and Dual-Level Reflection for Long-Horizon Embodied VLNcited by 1
  7. 2025Intelligent Design 4.0: Paradigm Evolution Toward the Agentic AI Eracited by 0
  8. 2026Nonuniformity Principle in Human-AI Coworkingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026An Empirical Study of Multi-Agent Collaboration for Automated Research
  2. 2026Security Considerations for Multi-agent Systems
  3. 2026When Do Multi-Agent Systems Help? An Information Bottleneck Perspective
  4. 2024AgentMonitor: A Plug-and-Play Framework for Predictive and Secure Multi-Agent Systems
  5. 2025Single-Agent Scaling Fails Multi-Agent Intelligence: Towards Foundation Models with Native Multi-Agent Intelligence
  6. 2026Towards a Science of Collective AI: LLM-based Multi-Agent Systems Need a Transition from Blind Trial-and-Error to Rigorous Science
  7. 2025Adaptive Monitoring and Real-World Evaluation of Agentic AI Systems
  8. 2026Understanding Agent Scaling in LLM-Based Multi-Agent Systems via Diversity