# A Reinforcement Learning Engine with Reduced Action and State Space for Scalable Cyber-Physical Optimal Response

arXiv: https://arxiv.org/abs/2410.04518 · HTML: https://mlsanity.com/paper/2410.04518/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 57 references that are in the map.
- First posted on arXiv 6 Oct 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [A Theoretical Analysis of Deep Q-Learning](https://mlsanity.com/paper/1901.00137/) (2019)
- [Reinforcement Learning for IoT Security: A Comprehensive Survey](https://mlsanity.com/paper/2102.07247/) (2021)
- [PowerGym: A Reinforcement Learning Environment for Volt-Var Control in Power Distribution Systems](https://mlsanity.com/paper/2109.03970/) (2021)
- [Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training](https://mlsanity.com/paper/2110.08956/) (2021)
- [Multi-Source Data Fusion for Cyberattack Detection in Power Systems](https://mlsanity.com/paper/2101.06897/) (2021)
- [Considerations in the Automatic Development of Electric Grid Restoration Plans](https://mlsanity.com/paper/2010.16035/) (2020)
- [Real-time Power System Simulation with Hardware Devices through DNP3 in Cyber-Physical Testbed](https://mlsanity.com/paper/2101.05965/) (2021)
- [Reinforcement Learning for Resilient Power Grids](https://mlsanity.com/paper/2212.04069/) (2022)

## Abstract (arXiv)

Numerous research studies have been conducted to enhance the resilience of cyber-physical systems (CPSs) by detecting potential cyber or physical disturbances. However, the development of scalable and optimal response measures under power system contingency based on fusing cyber-physical data is still in an early stage. To address this research gap, this paper introduces a power system response engine based on reinforcement learning (RL) and role and interaction discovery (RID) techniques. RL-RID-GridResponder is designed to automatically detect the contingency and assist with the decision-making process to ensure optimal power system operation. The RL-RID-GridResponder learns via an RL-based structure and achieves enhanced scalability by integrating an RID module with reduced action and state spaces. The applicability of RL-RID-GridResponder in providing scalable and optimal responses for CPSs is demonstrated on power systems in the context of Denial of Service (DoS) attacks. Moreover, simulations are conducted on a Volt-Var regulation problem using the augmented WSCC 9-bus and augmented IEEE 24-bus systems based on fused cyber and physical data sets. The results show that the proposed RL-RID-GridResponder can provide fast and accurate responses to ensure optimal power system operation under DoS and can extend to other system contingencies such as line outages and loss of loads.