# Reinforcement Learning for Feedback-Enabled Cyber Resilience

arXiv: https://arxiv.org/abs/2107.00783 · HTML: https://mlsanity.com/paper/2107.00783/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 178 references that are in the map.
- First posted on arXiv 2 Jul 2021.

## Builds on (most cited first, 30 of 52)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning](https://mlsanity.com/paper/1708.02596/) (2017)
- [An Introduction to Deep Reinforcement Learning](https://mlsanity.com/paper/1811.12560/) (2018)
- [Imagination-Augmented Agents for Deep Reinforcement Learning](https://mlsanity.com/paper/1707.06203/) (2017)
- [Deep Reinforcement Learning for Cyber Security](https://mlsanity.com/paper/1906.05799/) (2019)
- [Adaptive Reward-Poisoning Attacks against Reinforcement Learning](https://mlsanity.com/paper/2003.12613/) (2020)
- [Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learning](https://mlsanity.com/paper/2003.12909/) (2020)
- [Reinforcement Learning with Perturbed Rewards](https://mlsanity.com/paper/1810.01032/) (2018)
- [Policy Poisoning in Batch Reinforcement Learning and Control](https://mlsanity.com/paper/1910.05821/) (2019)
- [Reinforcement Learning with Random Delays](https://mlsanity.com/paper/2010.02966/) (2020)
- [Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RL](https://mlsanity.com/paper/2106.05087/) (2021)
- [Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signals](https://mlsanity.com/paper/1906.10571/) (2019)
- [Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamics](https://mlsanity.com/paper/2009.00774/) (2020)
- [Learning Adversarial Markov Decision Processes with Delayed Feedback](https://mlsanity.com/paper/2012.14843/) (2020)
- [DRL-FAS: A Novel Framework Based on Deep Reinforcement Learning for Face Anti-Spoofing](https://mlsanity.com/paper/2009.07529/) (2020)
- [Reinforcement Learning for IoT Security: A Comprehensive Survey](https://mlsanity.com/paper/2102.07247/) (2021)
- [A Game-Theoretic Taxonomy and Survey of Defensive Deception for Cybersecurity and Privacy](https://mlsanity.com/paper/1712.05441/) (2017)
- [IoT Security Techniques Based on Machine Learning](https://mlsanity.com/paper/1801.06275/) (2018)
- [Defense Against Reward Poisoning Attacks in Reinforcement Learning](https://mlsanity.com/paper/2102.05776/) (2021)
- [Falsification of Cyber-Physical Systems Using Deep Reinforcement Learning](https://mlsanity.com/paper/1805.00200/) (2018)
- [Robust Deep Reinforcement Learning for Security and Safety in Autonomous Vehicle Systems](https://mlsanity.com/paper/1805.00983/) (2018)
- [A Dynamic Games Approach to Proactive Defense Strategies against Advanced Persistent Threats in Cyber-Physical Systems](https://mlsanity.com/paper/1906.09687/) (2019)
- [Adversarial attacks in consensus-based multi-agent reinforcement learning](https://mlsanity.com/paper/2103.06967/) (2021)
- [Online Cyber-Attack Detection in Smart Grid: A Reinforcement Learning Approach](https://mlsanity.com/paper/1809.05258/) (2018)
- [Multi-agent Reinforcement Learning in Bayesian Stackelberg Markov Games for Adaptive Moving Target Defense](https://mlsanity.com/paper/2007.10457/) (2020)

## Built on by (most cited first, 19 of 19)

- [Automated Cyber Defence: A Review](https://mlsanity.com/paper/2303.04926/) (2023)
- [Intrusion Prevention through Optimal Stopping](https://mlsanity.com/paper/2111.00289/) (2021)
- [Learning Near-Optimal Intrusion Responses Against Dynamic Attackers](https://mlsanity.com/paper/2301.06085/) (2023)
- [Optimal Defender Strategies for CAGE-2 using Causal Modeling and Tree Search](https://mlsanity.com/paper/2407.11070/) (2024)
- [Canaries and Whistles: Resilient Drone Communication Networks with (or without) Deep Reinforcement Learning](https://mlsanity.com/paper/2312.04940/) (2023)
- [Scalable Learning of Intrusion Responses through Recursive Decomposition](https://mlsanity.com/paper/2309.03292/) (2023)
- [Intrusion Tolerance for Networked Systems through Two-Level Feedback Control](https://mlsanity.com/paper/2404.01741/) (2024)
- [Cooperative Resilience in Artificial Intelligence Multiagent Systems](https://mlsanity.com/paper/2409.13187/) (2024)
- [Learning Security Strategies through Game Play and Optimal Stopping](https://mlsanity.com/paper/2205.14694/) (2022)
- [A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions](https://mlsanity.com/paper/2508.09128/) (2025)
- [Training Automated Defense Strategies Using Graph-based Cyber Attack Simulations](https://mlsanity.com/paper/2304.11084/) (2023)
- [Adaptive Anomaly Detection for Identifying Attacks in Cyber-Physical Systems: A Systematic Literature Review](https://mlsanity.com/paper/2411.14278/) (2024)
- [Combating Informational Denial-of-Service (IDoS) Attacks: Modeling and Mitigation of Attentional Human Vulnerability](https://mlsanity.com/paper/2108.08255/) (2021)
- [Reinforcement Learning for Linear Quadratic Control is Vulnerable Under Cost Manipulation](https://mlsanity.com/paper/2203.05774/) (2022)
- [Resilient Constrained Reinforcement Learning](https://mlsanity.com/paper/2312.17194/) (2023)
- [Quantitative Resilience Modeling for Autonomous Cyber Defense](https://mlsanity.com/paper/2503.02780/) (2025)
- [ARM-IRL: Adaptive Resilience Metric Quantification Using Inverse Reinforcement Learning](https://mlsanity.com/paper/2501.12362/) (2025)
- [Cyber Deception for Mission Surveillance via Hypergame-Theoretic Deep Reinforcement Learning](https://mlsanity.com/paper/2603.20981/) (2026)
- [Inverting Self-Triggered Control: Adversarial Reinforcement Learning for Sparse Denial-of-Service Attacks](https://mlsanity.com/paper/2609.12016/) (2026)

## Abstract (arXiv)

Digitization and remote connectivity have enlarged the attack surface and made cyber systems more vulnerable. As attackers become increasingly sophisticated and resourceful, mere reliance on traditional cyber protection, such as intrusion detection, firewalls, and encryption, is insufficient to secure the cyber systems. Cyber resilience provides a new security paradigm that complements inadequate protection with resilience mechanisms. A Cyber-Resilient Mechanism (CRM) adapts to the known or zero-day threats and uncertainties in real-time and strategically responds to them to maintain critical functions of the cyber systems in the event of successful attacks. Feedback architectures play a pivotal role in enabling the online sensing, reasoning, and actuation process of the CRM. Reinforcement Learning (RL) is an essential tool that epitomizes the feedback architectures for cyber resilience. It allows the CRM to provide sequential responses to attacks with limited or without prior knowledge of the environment and the attacker. In this work, we review the literature on RL for cyber resilience and discuss cyber resilience against three major types of vulnerabilities, i.e., posture-related, information-related, and human-related vulnerabilities. We introduce three application domains of CRMs: moving target defense, defensive cyber deception, and assistive human security technologies. The RL algorithms also have vulnerabilities themselves. We explain the three vulnerabilities of RL and present attack models where the attacker targets the information exchanged between the environment and the agent: the rewards, the state observations, and the action commands. We show that the attacker can trick the RL agent into learning a nefarious policy with minimum attacking effort. Lastly, we discuss the future challenges of RL for cyber security and resilience and emerging applications of RL-based CRMs.