# Emergence in Multi-Agent Systems: A Safety Perspective

arXiv: https://arxiv.org/abs/2408.04514 · HTML: https://mlsanity.com/paper/2408.04514/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Aug 2024.

## Built on by (most cited first, 5 of 5)

- [A Survey on AgentOps: Categorization, Challenges, and Future Directions](https://mlsanity.com/paper/2508.02121/) (2025)
- [Convergence dynamics of Agent-to-Agent Interactions with Misaligned objectives](https://mlsanity.com/paper/2511.08710/) (2025)
- [Distributed Legal Infrastructure for a Trustworthy Agentic Web](https://mlsanity.com/paper/2603.06884/) (2026)
- [Agent System Operations: Categorization, Challenges, and Future Directions](https://mlsanity.com/paper/2606.01581/) (2026)
- [AI Assurance in UK Defence: Challenges in Operationalising JSP 936](https://mlsanity.com/paper/2606.09414/) (2026)

## Abstract (arXiv)

Emergent effects can arise in multi-agent systems (MAS) where execution is decentralized and reliant on local information. These effects may range from minor deviations in behavior to catastrophic system failures. To formally define these effects, we identify misalignments between the global inherent specification (the true specification) and its local approximation (such as the configuration of different reward components or observations). Using established safety terminology, we develop a framework to understand these emergent effects. To showcase the resulting implications, we use two broadly configurable exemplary gridworld scenarios, where insufficient specification leads to unintended behavior deviations when derived independently. Recognizing that a global adaptation might not always be feasible, we propose adjusting the underlying parameterizations to mitigate these issues, thereby improving the system's alignment and reducing the risk of emergent failures.