# Make Safe Decisions in Power System: Safe Reinforcement Learning Based Pre-decision Making for Voltage Stability Emergency Control

arXiv: https://arxiv.org/abs/2405.16485 · HTML: https://mlsanity.com/paper/2405.16485/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 34 references that are in the map.
- First posted on arXiv 26 May 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 11 of 11)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Lyapunov-based Safe Policy Optimization for Continuous Control](https://mlsanity.com/paper/1901.10031/) (2019)
- [Conservative Safety Critics for Exploration](https://mlsanity.com/paper/2010.14497/) (2020)
- [State-wise Safe Reinforcement Learning: A Survey](https://mlsanity.com/paper/2302.03122/) (2023)
- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [Evaluating Model-free Reinforcement Learning toward Safety-critical Tasks](https://mlsanity.com/paper/2212.05727/) (2022)
- [Stability Constrained Reinforcement Learning for Real-Time Voltage Control](https://mlsanity.com/paper/2109.14854/) (2021)
- [Safe Reinforcement Learning for Emergency LoadShedding of Power Systems](https://mlsanity.com/paper/2011.09664/) (2020)

## Abstract (arXiv)

The high penetration of renewable energy and power electronic equipment bring significant challenges to the efficient construction of adaptive emergency control strategies against various presumed contingencies in today's power systems. Traditional model-based emergency control methods have difficulty in adapt well to various complicated operating conditions in practice. Fr emerging artificial intelligence-based approaches, i.e., reinforcement learning-enabled solutions, they are yet to provide solid safety assurances under strict constraints in practical power systems. To address these research gaps, this paper develops a safe reinforcement learning (SRL)-based pre-decision making framework against short-term voltage collapse. Our proposed framework employs neural networks for pre-decision formulation, security margin estimation, and corrective action implementation, without reliance on precise system parameters. Leveraging the gradient projection, we propose a security projecting correction algorithm that offers theoretical security assurances to amend risky actions. The applicability of the algorithm is further enhanced through the incorporation of active learning, which expedites the training process and improves security estimation accuracy. Extensive numerical tests on the New England 39-bus system and the realistic Guangdong Provincal Power Grid demonstrate the effectiveness of the proposed framework.