Training RL Agents for Multi-Objective Network Defense Tasks
Not on the map. Why?
Abstract (arXiv)
Open-ended learning (OEL) -- which emphasizes training agents that achieve broad capability over narrow competency -- is emerging as a paradigm to develop artificial intelligence (AI) agents to achieve robustness and generalization. However, despite promising results that demonstrate the benefits of OEL, applying OEL to develop autonomous agents for real-world cybersecurity applications remains a challenge. We propose a training approach, inspired by OEL, to develop autonomous network defenders. Our results demonstrate that like in other domains, OEL principles can translate into more robust and generalizable agents for cyber defense. To apply OEL to network defense, it is necessary to address several technical challenges. Most importantly, it is critical to provide a task representation approach over a broad universe of tasks that maintains a consistent interface over goals, rewards and action spaces. This way, the learning agent can train with varying network conditions, attacker behaviors, and defender goals while being able to build on previously gained knowledge. With our tools and results, we aim to fundamentally impact research that applies AI to solve cybersecurity problems. Specifically, as researchers develop gyms and benchmarks for cyber defense, it is paramount that they consider diverse tasks with consistent representations, such as those we propose in our work.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 26 of its 44 references that are in the map. First posted on arXiv 28 May 2025.
Builds on
26 of 44 references.
References · 26
- 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 752
- 2019Solving Rubik's Cube with a Robot Handcited by 734
- 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
- 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 313
- 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
- 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
- 2019Online Meta-Learningcited by 137
- 2019Open-ended Learning in Symmetric Zero-sum Gamescited by 96
- 2019Learning Action Representations for Reinforcement Learningcited by 75
- 2021CybORG: A Gym for the Development of Autonomous Cyber Agentscited by 60
- 2021Modelling Behavioural Diversity for Learning in Open-Ended Gamescited by 39
- 2022Developing Optimal Causal Cyber-Defence Agents via Cyber Security Simulationcited by 37
14 more
- 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 33
- 2021Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systemscited by 24
- 2023NASimEmu: Network Attack Simulator & Emulator for Training Agents Generalizing to Novel Scenarioscited by 20
- 2021Deep hierarchical reinforcement agents for automated penetration testingcited by 19
- 2023Reward Shaping for Happier Autonomous Cyber Security Agentscited by 19
- 2020CybORG: An Autonomous Cyber Operations Research Gymcited by 16
- 2021Using Cyber Terrain in Reinforcement Learning for Penetration Testingcited by 16
- 2018Online Robust Policy Learning in the Presence of Unknown Adversariescited by 15
- 2023Learning Near-Optimal Intrusion Responses Against Dynamic Attackerscited by 15
- 2021CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systemscited by 11
- 2019Rogue-Gym: A New Challenge for Generalization in Reinforcement Learningcited by 6
- 2021Network Defense is Not a Gamecited by 6
- 2023In Search of netUnicorn: A Data-Collection Platform to Develop Generalizable ML Models for Network Security Problemscited by 6
- 2023Enabling A Network AI Gym for Autonomous Cyber Agentscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.