mlsanityData to 14 Oct 2026

Building Better Environments for Autonomous Cyber Defence

arXiv 2604.08805 ↗First posted 9 Apr 2026cs.CR cs.AIcited by 3

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

In November 2025, the authors ran a workshop on the topic of what makes a good reinforcement learning (RL) environment for autonomous cyber defence (ACD). This paper details the knowledge shared by participants both during the workshop and shortly afterwards by contributing herein. The workshop participants come from academia, industry, and government, and have extensive hands-on experience designing and working with RL and cyber environments. While there is now a sizeable body of literature describing work in RL for ACD, there is nevertheless a great deal of tradecraft, domain knowledge, and common hazards which are not detailed comprehensively in a single resource. With a specific focus on building better environments to train and evaluate autonomous RL agents in network defence scenarios, including government and critical infrastructure networks, the contributions of this work are twofold: (1) a framework for decomposing the interface between RL cyber environments and real systems, and (2) guidelines on current best practice for RL-based ACD environment development and agent evaluation, based on the key findings from our workshop.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 35 of its 69 references that are in the map. First posted on arXiv 9 Apr 2026.

Builds on

35 of 69 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity9 Feb
  2. 2026Beyond Rewards in Reinforcement Learning for Cyber Defence4 Feb
  3. 2025Towards a Generalisable Cyber Defence Agent for Real-World Computer Networks12 Nov
  4. 2025Automated Cyber Defense with Generalizable Graph-based Reinforcement Learning Agents19 Sep
  5. 2025DRMD: Deep Reinforcement Learning for Malware Detection under Concept Drift26 Aug
  6. 2025Less is more? Rewards in RL for Cyber Defence5 Mar
  7. 2024Entity-based Reinforcement Learning for Autonomous Cyber Defence23 Oct
  8. 2024CybORG++: An Enhanced Gym for the Development of Autonomous Cyber Agents18 Oct
  9. 2024Autonomous Network Defence using Reinforcement Learning26 Sep
  10. 2024Structural Generalization in Autonomous Cyber Incident Response with Message-Passing Neural Networks and Reinforcement Learning8 Jul
  11. 2024Optimizing Cyber Defense in Dynamic Active Directories through Reinforcement Learning28 Jun

Other references · 24

  1. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  2. 2018Assessing Generalization in Deep Reinforcement Learningcited by 133
  3. 2021CybORG: A Gym for the Development of Autonomous Cyber Agentscited by 60
  4. 2023Empirical Design in Reinforcement Learningcited by 55
  5. 2022Developing Optimal Causal Cyber-Defence Agents via Cyber Security Simulationcited by 37
  6. 2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 33
  7. 2021Network Environment Design for Autonomous Cyberdefensecited by 32
  8. 2023On Autonomous Agents in a Cyber Defence Environmentcited by 26
  9. 2024A Survey on Self-play Methods in Reinforcement Learningcited by 26
  10. 2019Scalable Reinforcement Learning for Multi-Agent Networked Systemscited by 24
  11. 2023NASimEmu: Network Attack Simulator & Emulator for Training Agents Generalizing to Novel Scenarioscited by 20
  12. 2023Reward Shaping for Happier Autonomous Cyber Security Agentscited by 19
12 more
  1. 2021Intrusion Prevention through Optimal Stoppingcited by 18
  2. 2023Inroads into Autonomous Network Defence using Explained Reinforcement Learningcited by 17
  3. 2021Using Cyber Terrain in Reinforcement Learning for Penetration Testingcited by 16
  4. 2022ATTRITION: Attacking Static Hardware Trojan Detection Techniques Using Reinforcement Learningcited by 11
  5. 2022AutoCAT: Reinforcement Learning for Automated Exploration of Cache-Timing Attackscited by 9
  6. 2023Canaries and Whistles: Resilient Drone Communication Networks with (or without) Deep Reinforcement Learningcited by 9
  7. 2023Deep PackGen: A Deep Reinforcement Learning Framework for Adversarial Network Packet Generationcited by 6
  8. 2020Exchangeable Input Representations for Reinforcement Learningcited by 5
  9. 2020Causal Campbell-Goodhart's law and Reinforcement Learningcited by 5
  10. 2024APIRL: Deep Reinforcement Learning for REST API Fuzzingcited by 4
  11. 2024Out-of-Distribution Detection for Neurosymbolic Autonomous Cyber Agentscited by 3
  12. 2025Training RL Agents for Multi-Objective Network Defense Taskscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026REPLICANT: Learning Policies for Evading and Hardening Malware Detectorscited by 0
  2. 2026A Cyber Range Evaluation of Autonomous Network Incident Response Agentscited by 0
  3. 2026The MAL Simulator: Cyber Operations Simulation based on Attack & Defense Graphscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Guidelines for Applying RL and MARL in Cybersecurity Applications
  2. 2026Distilling Knowledge from Large Language Models into Lightweight Reinforcement Learning Agents for Autonomous Cyber Operations
  3. 2024Exploring reinforcement learning for incident response in autonomous military vehicles
  4. 2025Towards Production-Worthy Simulation for Autonomous Cyber Operations
  5. 2025An Empirical Game-Theoretic Analysis of Autonomous Cyber-Defence Agents
  6. 2025Multi-Agent Reinforcement Learning in Cybersecurity: From Fundamentals to Applications
  7. 2026Ask the Expert: LLM-Guided Reinforcement Learning for Autonomous Cyber Defense
  8. 2025Quantitative Resilience Modeling for Autonomous Cyber Defense