# Canaries and Whistles: Resilient Drone Communication Networks with (or without) Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2312.04940 · HTML: https://mlsanity.com/paper/2312.04940/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 60 references that are in the map.
- First posted on arXiv 8 Dec 2023.

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games](https://mlsanity.com/paper/2103.01955/) (2021)
- [Emergent Tool Use From Multi-Agent Autocurricula](https://mlsanity.com/paper/1909.07528/) (2019)
- [PettingZoo: Gym for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2009.14471/) (2020)
- [Mean Field Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1802.05438/) (2018)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [CybORG: A Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2108.09118/) (2021)
- [Developing Optimal Causal Cyber-Defence Agents via Cyber Security Simulation](https://mlsanity.com/paper/2207.12355/) (2022)
- [From Few to More: Large-scale Dynamic Multiagent Curriculum Learning](https://mlsanity.com/paper/1909.02790/) (2019)
- [Finding Effective Security Strategies through Reinforcement Learning and Self-Play](https://mlsanity.com/paper/2009.08120/) (2020)
- [Beyond CAGE: Investigating Generalization of Learned Autonomous Network Defense Policies](https://mlsanity.com/paper/2211.15557/) (2022)
- [Reinforcement Learning for Feedback-Enabled Cyber Resilience](https://mlsanity.com/paper/2107.00783/) (2021)
- [Reinforcement Learning for Autonomous Defence in Software-Defined Networking](https://mlsanity.com/paper/1808.05770/) (2018)
- [Inroads into Autonomous Network Defence using Explained Reinforcement Learning](https://mlsanity.com/paper/2306.09318/) (2023)
- [ATTRITION: Attacking Static Hardware Trojan Detection Techniques Using Reinforcement Learning](https://mlsanity.com/paper/2208.12897/) (2022)
- [A Survey on Security and Privacy Issues of UAVs](https://mlsanity.com/paper/2109.14442/) (2021)
- [AutoCAT: Reinforcement Learning for Automated Exploration of Cache-Timing Attacks](https://mlsanity.com/paper/2208.08025/) (2022)
- [Building Embedded Systems Like It's 1996](https://mlsanity.com/paper/2203.06834/) (2022)

## Built on by (most cited first, 9 of 9)

- [CybORG++: An Enhanced Gym for the Development of Autonomous Cyber Agents](https://mlsanity.com/paper/2410.16324/) (2024)
- [Entity-based Reinforcement Learning for Autonomous Cyber Defence](https://mlsanity.com/paper/2410.17647/) (2024)
- [DRMD: Deep Reinforcement Learning for Malware Detection under Concept Drift](https://mlsanity.com/paper/2508.18839/) (2025)
- [Optimizing Cyber Defense in Dynamic Active Directories through Reinforcement Learning](https://mlsanity.com/paper/2406.19596/) (2024)
- [Online Incident Response Planning under Model Misspecification through Bayesian Learning and Belief Quantization](https://mlsanity.com/paper/2508.14385/) (2025)
- [SoK: The Pitfalls of Deep Reinforcement Learning for Cybersecurity](https://mlsanity.com/paper/2602.08690/) (2026)
- [Beyond Rewards in Reinforcement Learning for Cyber Defence](https://mlsanity.com/paper/2602.04809/) (2026)
- [Building Better Environments for Autonomous Cyber Defence](https://mlsanity.com/paper/2604.08805/) (2026)
- [Inherently Interpretable and Uncertainty-Aware Models for Online Learning in Cyber-Security Problems](https://mlsanity.com/paper/2411.09393/) (2024)

## Abstract (arXiv)

Communication networks able to withstand hostile environments are critically important for disaster relief operations. In this paper, we consider a challenging scenario where drones have been compromised in the supply chain, during their manufacture, and harbour malicious software capable of wide-ranging and infectious disruption. We investigate multi-agent deep reinforcement learning as a tool for learning defensive strategies that maximise communications bandwidth despite continual adversarial interference. Using a public challenge for learning network resilience strategies, we propose a state-of-the-art expert technique and study its superiority over deep reinforcement learning agents. Correspondingly, we identify three specific methods for improving the performance of our learning-based agents: (1) ensuring each observation contains the necessary information, (2) using expert agents to provide a curriculum for learning, and (3) paying close attention to reward. We apply our methods and present a new mixed strategy enabling expert and learning-based agents to work together and improve on all prior results.