mlsanityData to 14 Oct 2026

PoolFlip: A Multi-Agent Reinforcement Learning Security Environment for Cyber Defense

arXiv 2508.19488 ↗First posted 27 Aug 2025cs.LG cs.AI cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

Cyber defense requires automating defensive decision-making under stealthy, deceptive, and continuously evolving adversarial strategies. The FlipIt game provides a foundational framework for modeling interactions between a defender and an advanced adversary that compromises a system without being immediately detected. In FlipIt, the attacker and defender compete to control a shared resource by performing a Flip action and paying a cost. However, the existing FlipIt frameworks rely on a small number of heuristics or specialized learning techniques, which can lead to brittleness and the inability to adapt to new attacks. To address these limitations, we introduce PoolFlip, a multi-agent gym environment that extends the FlipIt game to allow efficient learning for attackers and defenders. Furthermore, we propose Flip-PSRO, a multi-agent reinforcement learning (MARL) approach that leverages population-based training to train defender agents equipped to generalize against a range of unknown, potentially adaptive opponents. Our empirical results suggest that Flip-PSRO defenders are 2× more effective than baselines to generalize to a heuristic attack not exposed in training. In addition, our newly designed ownership-based utility functions ensure that Flip-PSRO defenders maintain a high level of control while optimizing performance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 48 references that are in the map. First posted on arXiv 27 Aug 2025.

Builds on

21 of 48 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2017A Unified Game-Theoretic Approach to Multiagent Reinforcement Learningcited by 268
  4. 2019Deep Reinforcement Learning for Cyber Securitycited by 95
  5. 2021CybORG: A Gym for the Development of Autonomous Cyber Agentscited by 60
  6. 2020Pipeline PSRO: A Scalable Approach for Finding Approximate Nash Equilibria in Large Gamescited by 47
  7. 2023On Autonomous Agents in a Cyber Defence Environmentcited by 26
  8. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  9. 2018Deep Reinforcement Learning for Green Security Games with Real-Time Informationcited by 19
  10. 2023Evolving Diverse Red-team Language Models in Multi-round Multi-agent Gamescited by 17
  11. 2024Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defensecited by 17
  12. 2023Policy Space Diversity for Non-Transitive Gamescited by 15
9 more
  1. 2019Finding Needles in a Moving Haystack: Prioritizing Alerts with Adversarial Reinforcement Learningcited by 12
  2. 2021Robust Reinforcement Learning Under Minimax Regret for Green Securitycited by 12
  3. 2023Learning Cyber Defence Tactics from Scratch with Multi-Agent Reinforcement Learningcited by 12
  4. 2024Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Searchcited by 12
  5. 2024Policy Space Response Oracles: A Surveycited by 10
  6. 2020Learning to Play against Any Mixture of Opponentscited by 8
  7. 2023Computing Optimal Equilibria and Mechanisms via Learning in Zero-Sum Extensive-Form Gamescited by 6
  8. 2024Fusion-PSRO: Nash Policy Fusion for Policy Space Response Oraclescited by 2
  9. 2020Deep Reinforcement Learning for FlipIt Security Gamecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Multi-Agent Reinforcement Learning in Cybersecurity: From Fundamentals to Applications
  2. 2025Nash Q-Network for Multi-Agent Cybersecurity Simulation
  3. 2025Learning to Communicate in Multi-Agent Reinforcement Learning for Autonomous Cyber Defence
  4. 2026Distilling Knowledge from Large Language Models into Lightweight Reinforcement Learning Agents for Autonomous Cyber Operations
  5. 2025Automated Cyber Defense with Generalizable Graph-based Reinforcement Learning Agents
  6. 2026Towards Hierarchical Cyber Defense with Large Language Models: From Planning to Execution
  7. 2025Interpreting Agent Behaviors in Reinforcement-Learning-Based Cyber-Battle Simulation Platforms
  8. 2024SUB-PLAY: Adversarial Policies against Partially Observed Multi-Agent Reinforcement Learning Systems