# Multi-Agent Reinforcement Learning in Cybersecurity: From Fundamentals to Applications

arXiv: https://arxiv.org/abs/2505.19837 · HTML: https://mlsanity.com/paper/2505.19837/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 70 references that are in the map.
- First posted on arXiv 26 May 2025.

## Builds on (most cited first, 27 of 27)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments](https://mlsanity.com/paper/1706.02275/) (2017)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Counterfactual Multi-Agent Policy Gradients](https://mlsanity.com/paper/1705.08926/) (2017)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [The StarCraft Multi-Agent Challenge](https://mlsanity.com/paper/1902.04043/) (2019)
- [Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://mlsanity.com/paper/1706.05296/) (2017)
- [Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?](https://mlsanity.com/paper/2011.09533/) (2020)
- [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2009.14471/) (2020)
- [OpenSpiel: A Framework for Reinforcement Learning in Games](https://mlsanity.com/paper/1908.09453/) (2019)
- [Multiagent Cooperation and Competition with Deep Reinforcement Learning](https://mlsanity.com/paper/1511.08779/) (2015)
- [Near Optimal Behavior via Approximate State Abstraction](https://mlsanity.com/paper/1701.04113/) (2017)
- [Multi-agent Reinforcement Learning: A Comprehensive Survey](https://mlsanity.com/paper/2312.10256/) (2023)
- [A Theory of Abstraction in Reinforcement Learning](https://mlsanity.com/paper/2203.00397/) (2022)
- [Reinforcement Learning for IoT Security: A Comprehensive Survey](https://mlsanity.com/paper/2102.07247/) (2021)
- [MARLlib: A Scalable and Efficient Multi-agent Reinforcement Learning Library](https://mlsanity.com/paper/2210.13708/) (2022)
- [Automated Cyber Defence: A Review](https://mlsanity.com/paper/2303.04926/) (2023)
- [NASimEmu: Network Attack Simulator & Emulator for Training Agents Generalizing to Novel Scenarios](https://mlsanity.com/paper/2305.17246/) (2023)
- [CybORG: An Autonomous Cyber Operations Research Gym](https://mlsanity.com/paper/2002.10667/) (2020)
- [CybORG++: An Enhanced Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2410.16324/) (2024)
- [Autonomous Intelligent Cyber-defense Agent (AICA) Reference Architecture. Release 2.0](https://mlsanity.com/paper/1803.10664/) (2018)
- [Deep Q-Learning based Reinforcement Learning Approach for Network Intrusion Detection](https://mlsanity.com/paper/2111.13978/) (2021)
- [A Multiagent CyberBattleSim for RL Cyber Operation Agents](https://mlsanity.com/paper/2304.11052/) (2023)
- [Deep Reinforcement Learning for Autonomous Cyber Defence: A Survey](https://mlsanity.com/paper/2310.07745/) (2023)
- [Combating Advanced Persistent Threats: Challenges and Solutions](https://mlsanity.com/paper/2309.09498/) (2023)
- [Multi-Agent Reinforcement Learning for Maritime Operational Technology Cyber Security](https://mlsanity.com/paper/2401.10149/) (2024)
- [A Survey for Deep Reinforcement Learning Based Network Intrusion Detection](https://mlsanity.com/paper/2410.07612/) (2024)

## Built on by (most cited first, 1 of 1)

- [A Survey of Agentic AI and Cybersecurity: Challenges, Opportunities and Use-case Prototypes](https://mlsanity.com/paper/2601.05293/) (2026)

## Abstract (arXiv)

Multi-Agent Reinforcement Learning (MARL) has shown great potential as an adaptive solution for addressing modern cybersecurity challenges. MARL enables decentralized, adaptive, and collaborative defense strategies and provides an automated mechanism to combat dynamic, coordinated, and sophisticated threats. This survey investigates the current state of research in MARL applications for automated cyber defense (ACD), focusing on intruder detection and lateral movement containment. Additionally, it examines the role of Autonomous Intelligent Cyber-defense Agents (AICA) and Cyber Gyms in training and validating MARL agents. Finally, the paper outlines existing challenges, such as scalability and adversarial robustness, and proposes future research directions. This also discusses how MARL integrates in AICA to provide adaptive, scalable, and dynamic solutions to counter the increasingly sophisticated landscape of cyber threats. It highlights the transformative potential of MARL in areas like intrusion detection and lateral movement containment, and underscores the value of Cyber Gyms for training and validation of AICA.