# Computationally Efficient Safe Reinforcement Learning for Power Systems

arXiv: https://arxiv.org/abs/2110.10333 · HTML: https://mlsanity.com/paper/2110.10333/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2021.

## Built on by (most cited first, 20 of 20)

- [Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarking](https://mlsanity.com/paper/2205.06750/) (2022)
- [Excluding the Irrelevant: Focusing Reinforcement Learning through Continuous Action Masking](https://mlsanity.com/paper/2406.03704/) (2024)
- [A Review of Safe Reinforcement Learning Methods for Modern Power Systems](https://mlsanity.com/paper/2407.00304/) (2024)
- [Enforcing Hard Linear Constraints in Deep Learning Models with Decision Rules](https://mlsanity.com/paper/2505.13858/) (2025)
- [Enhancing Efficiency of Safe Reinforcement Learning via Sample Manipulation](https://mlsanity.com/paper/2405.20860/) (2024)
- [Differentiable Safe Controller Design through Control Barrier Functions](https://mlsanity.com/paper/2209.10034/) (2022)
- [GLinSAT: The General Linear Satisfiability Neural Network Layer By Accelerated Gradient Descent](https://mlsanity.com/paper/2409.17500/) (2024)
- [Safe Reinforcement Learning using Action Projection: Safeguard the Policy or the Environment?](https://mlsanity.com/paper/2509.12833/) (2025)
- [CommonPower: A Framework for Safe Data-Driven Smart Grid Control](https://mlsanity.com/paper/2406.03231/) (2024)
- [Control invariant set enhanced safe reinforcement learning: improved sampling efficiency, guaranteed stability and robustness](https://mlsanity.com/paper/2305.15602/) (2023)
- [Self-supervised Equality Embedded Deep Lagrange Dual for Approximate Constrained Optimization](https://mlsanity.com/paper/2306.06674/) (2023)
- [Provable Traffic Rule Compliance in Safe Reinforcement Learning on the Open Sea](https://mlsanity.com/paper/2402.08502/) (2024)
- [An Efficient Learning-Based Solver for Two-Stage DC Optimal Power Flow with Feasibility Guarantees](https://mlsanity.com/paper/2304.01409/) (2023)
- [Control invariant set enhanced reinforcement learning for process control: improved sampling efficiency and guaranteed stability](https://mlsanity.com/paper/2304.05509/) (2023)
- [A Critical Review of Safe Reinforcement Learning Techniques in Smart Grid Applications](https://mlsanity.com/paper/2409.16256/) (2024)
- [Leveraging Analytic Gradients in Provably Safe Reinforcement Learning](https://mlsanity.com/paper/2506.01665/) (2025)
- [Differentiable Policy Transport over Multi-Layer Network Feasibility Geometry](https://mlsanity.com/paper/2609.26068/) (2026)
- [Interpreting Primal-Dual Algorithms for Constrained Multiagent Reinforcement Learning](https://mlsanity.com/paper/2211.16069/) (2022)
- [Toward Rapid, Optimal, and Feasible Power Dispatch through Generalized Neural Mapping](https://mlsanity.com/paper/2311.04838/) (2023)
- [Safe Reinforcement Learning-based Automatic Generation Control](https://mlsanity.com/paper/2507.17868/) (2025)

## Abstract (arXiv)

We propose a computationally efficient approach to safe reinforcement learning (RL) for frequency regulation in power systems with high levels of variable renewable energy resources. The approach draws on set-theoretic control techniques to craft a neural network-based control policy that is guaranteed to satisfy safety-critical state constraints, without needing to solve a model predictive control or projection problem in real time. By exploiting the properties of robust controlled-invariant polytopes, we construct a novel, closed-form "safety-filter" that enables end-to-end safe learning using any policy gradient-based RL algorithm. We then apply the safety filter in conjunction with the deep deterministic policy gradient (DDPG) algorithm to regulate frequency in a modified 9-bus power system, and show that the learned policy is more cost-effective than robust linear feedback control techniques while maintaining the same safety guarantee. We also show that the proposed paradigm outperforms DDPG augmented with constraint violation penalties.