# Demonstrations of Integrity Attacks in Multi-Agent Systems

arXiv: https://arxiv.org/abs/2506.04572 · HTML: https://mlsanity.com/paper/2506.04572/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use.
- Builds on 13 of its 22 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Placed in

- LLM agents and tool use / [Agent threats, guardrails and governance](https://mlsanity.com/agents/agent-threats-guardrails-and-governance/)

## Builds on (most cited first, 13 of 13)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate](https://mlsanity.com/paper/2305.19118/) (2023)
- [ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate](https://mlsanity.com/paper/2308.07201/) (2023)
- [Self-collaboration Code Generation via ChatGPT](https://mlsanity.com/paper/2304.07590/) (2023)
- [AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks](https://mlsanity.com/paper/2403.04783/) (2024)
- [Red-Teaming LLM Multi-Agent Systems via Communication Attacks](https://mlsanity.com/paper/2502.14847/) (2025)
- [PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety](https://mlsanity.com/paper/2401.11880/) (2024)
- [MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debate](https://mlsanity.com/paper/2406.14711/) (2024)
- [JAILJUDGE: A Comprehensive Jailbreak Judge Benchmark with Multi-Agent Enhanced Explanation Evaluation Framework](https://mlsanity.com/paper/2410.12855/) (2024)
- [AgentBreeder: Mitigating the AI Safety Risks of Multi-Agent Scaffolds via Self-Improvement](https://mlsanity.com/paper/2502.00757/) (2025)

## Built on by (most cited first, 8 of 8)

- [Aegis: Automated Error Generation and Attribution for Multi-Agent Systems](https://mlsanity.com/paper/2509.14295/) (2025)
- [Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation](https://mlsanity.com/paper/2606.10749/) (2026)
- [Securing Multi-Agent Systems Against Corruptions via Node Contribution Backpropagation](https://mlsanity.com/paper/2510.19420/) (2025)
- [Agent-Fence: Mapping Security Vulnerabilities Across Deep Research Agents](https://mlsanity.com/paper/2602.07652/) (2026)
- [CREBench: Evaluating Large Language Models in Cryptographic Binary Reverse Engineering](https://mlsanity.com/paper/2604.03750/) (2026)
- [Attacks and Mitigations for Distributed Governance of Agentic AI under Byzantine Adversaries](https://mlsanity.com/paper/2605.12364/) (2026)
- [VirtualCrime: Evaluating the Criminal Potential and Agentic Behaviors of Large Language Models via Sandbox Simulation](https://mlsanity.com/paper/2601.13981/) (2026)
- [Hierarchical Attacks for Multi-Modal Multi-Agent Reasoning](https://mlsanity.com/paper/2605.13213/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding, code generation, and complex planning. Simultaneously, Multi-Agent Systems (MAS) have garnered attention for their potential to enable cooperation among distributed agents. However, from a multi-party perspective, MAS could be vulnerable to malicious agents that exploit the system to serve self-interests without disrupting its core functionality. This work explores integrity attacks where malicious agents employ subtle prompt manipulation to bias MAS operations and gain various benefits. Four types of attacks are examined: textitScapegoater, who misleads the system monitor to underestimate other agents' contributions; textitBoaster, who misleads the system monitor to overestimate their own performance; textitSelf-Dealer, who manipulates other agents to adopt certain tools; and textitFree-Rider, who hands off its own task to others. We demonstrate that strategically crafted prompts can introduce systematic biases in MAS behavior and executable instructions, enabling malicious agents to effectively mislead evaluation systems and manipulate collaborative agents. Furthermore, our attacks can bypass advanced LLM-based monitors, such as GPT-4o-mini and o3-mini, highlighting the limitations of current detection mechanisms. Our findings underscore the critical need for MAS architectures with robust security protocols and content validation mechanisms, alongside monitoring systems capable of comprehensive risk scenario assessment.