# A Scalable Approach to Solving Simulation-Based Network Security Games

arXiv: https://arxiv.org/abs/2602.16564 · HTML: https://mlsanity.com/paper/2602.16564/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 29 references that are in the map.
- First posted on arXiv 18 Feb 2026.

## Placed in

- Deep reinforcement learning methods / [Adversarial robustness and RL in finance](https://mlsanity.com/deeprl/adversarial-robustness-and-rl-in-finance/)

## Builds on (most cited first, 10 of 10)

- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [FeUdal Networks for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1703.01161/) (2017)
- [Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?](https://mlsanity.com/paper/2011.09533/) (2020)
- [Deep Reinforcement Learning in Large Discrete Action Spaces](https://mlsanity.com/paper/1512.07679/) (2015)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [Hierarchical Multi-agent Reinforcement Learning for Cyber Network Defense](https://mlsanity.com/paper/2410.17351/) (2024)
- [Reinforcing Language Agents via Policy Optimization with Action Decomposition](https://mlsanity.com/paper/2405.15821/) (2024)
- [Stackelberg vs. Nash in Security Games: An Extended Investigation of Interchangeability, Equivalence, and Uniqueness](https://mlsanity.com/paper/1401.3888/) (2014)
- [CyGym: A Simulation-Based Game-Theoretic Analysis Framework for Cybersecurity](https://mlsanity.com/paper/2506.21688/) (2025)
- [Active Geospatial Search for Efficient Tenant Eviction Outreach](https://mlsanity.com/paper/2412.17854/) (2024)

## Built on by (most cited first, 1 of 1)

- [Efficient Nash Equilibrium Computation for Cybersecurity Games](https://mlsanity.com/paper/2609.19399/) (2026)

## Abstract (arXiv)

We introduce MetaDOAR, a lightweight meta-controller that augments the Double Oracle / PSRO paradigm with a learned, partition-aware filtering layer and Q-value caching to enable scalable multi-agent reinforcement learning on very large cyber-network environments. MetaDOAR learns a compact state projection from per node structural embeddings to rapidly score and select a small subset of devices (a top-k partition) on which a conventional low-level actor performs focused beam search utilizing a critic agent. Selected candidate actions are evaluated with batched critic forwards and stored in an LRU cache keyed by a quantized state projection and local action identifiers, dramatically reducing redundant critic computation while preserving decision quality via conservative k-hop cache invalidation. Empirically, MetaDOAR attains higher player payoffs than SOTA baselines on large network topologies, without significant scaling issues in terms of memory usage or training time. This contribution provide a practical, theoretically motivated path to efficient hierarchical policy learning for large-scale networked decision problems.