# Real-Time Defense Against Coordinated Cyber-Physical Attacks: A Robust Constrained Reinforcement Learning Approach

arXiv: https://arxiv.org/abs/2509.10999 · HTML: https://mlsanity.com/paper/2509.10999/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 39 references that are in the map.
- First posted on arXiv 13 Sep 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 9 of 9)

- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Distributionally Robust Chance Constrained Data-enabled Predictive Control](https://mlsanity.com/paper/2006.01702/) (2020)
- [Cyber-Security in Smart Grid: Survey and Challenges](https://mlsanity.com/paper/1809.02609/) (2018)
- [Adversarially Robust Learning for Security-Constrained Optimal Power Flow](https://mlsanity.com/paper/2111.06961/) (2021)
- [Deep Reinforcement Learning for DER Cyber-Attack Mitigation](https://mlsanity.com/paper/2009.13088/) (2020)
- [Optimal Attack against Cyber-Physical Control Systems with Reactive Attack Mitigation](https://mlsanity.com/paper/1706.01628/) (2017)
- [Performance and Resilience of Cyber-Physical Control Systems with Reactive Attack Mitigation](https://mlsanity.com/paper/1904.09445/) (2019)
- [Adam-based Augmented Random Search for Control Policies for Distributed Energy Resource Cyber Attack Mitigation](https://mlsanity.com/paper/2201.11825/) (2022)
- [Robust Optimal Power Flow Against Adversarial Attacks: A Tri-Level Optimization Approach](https://mlsanity.com/paper/2411.08618/) (2024)

## Abstract (arXiv)

Modern power systems face increasing vulnerability to sophisticated cyber-physical attacks beyond traditional N-1 contingency frameworks. Existing security paradigms face a critical bottleneck: efficiently identifying worst-case scenarios and rapidly coordinating defensive responses are hindered by intensive computation and time delays, during which cascading failures can propagate. This paper presents a novel tri-level robust constrained reinforcement learning (RCRL) framework for robust power system security. The framework generates diverse system states through AC-OPF formulations, identifies worst-case N-K attack scenarios for each state, and trains policies to mitigate these scenarios across all operating conditions without requiring predefined attack patterns. The framework addresses constraint satisfaction through Beta-blending projection-based feasible action mapping techniques during training and primal-dual augmented Lagrangian optimization for deployment. Once trained, the RCRL policy learns how to control observed cyber-physical attacks in real time. Validation on IEEE benchmark systems demonstrates effectiveness against coordinated N-K attacks, causing widespread cascading failures throughout the network. The learned policy can successfully respond rapidly to recover system-wide constraints back to normal within 0.21 ms inference times, establishing superior resilience for critical infrastructure protection.