# Reinforcement Learning for Optimal Primary Frequency Control: A Lyapunov Approach

arXiv: https://arxiv.org/abs/2009.05654 · HTML: https://mlsanity.com/paper/2009.05654/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Sep 2020.

## Built on by (most cited first, 14 of 14)

- [A Review of Safe Reinforcement Learning Methods for Modern Power Systems](https://mlsanity.com/paper/2407.00304/) (2024)
- [Reinforcement Learning with Elastic Time Steps](https://mlsanity.com/paper/2402.14961/) (2024)
- [Safe Reinforcement Learning for Power System Control: A Review](https://mlsanity.com/paper/2407.00681/) (2024)
- [Online Event-Triggered Switching for Frequency Control in Power Grids with Variable Inertia](https://mlsanity.com/paper/2408.15436/) (2024)
- [Optimal Design of Volt/VAR Control Rules of Inverters using Deep Learning](https://mlsanity.com/paper/2211.09557/) (2022)
- [Discovery of False Data Injection Schemes on Frequency Controllers with Reinforcement Learning](https://mlsanity.com/paper/2408.16958/) (2024)
- [Detection of False Data Injection Attacks (FDIA) on Power Dynamical Systems With a State Prediction Method](https://mlsanity.com/paper/2409.04609/) (2024)
- [Combining Neural Networks and Symbolic Regression for Analytical Lyapunov Function Discovery](https://mlsanity.com/paper/2406.15675/) (2024)
- [LEVIS: Large Exact Verifiable Input Spaces for Neural Networks](https://mlsanity.com/paper/2408.08824/) (2024)
- [Analytical Lyapunov Function Discovery: An RL-based Generative Approach](https://mlsanity.com/paper/2502.02014/) (2025)
- [Efficient Reinforcement Learning Through Trajectory Generation](https://mlsanity.com/paper/2211.17249/) (2022)
- [Continual Adversarial Reinforcement Learning (CARL) of False Data Injection detection: forgetting and explainability](https://mlsanity.com/paper/2411.10367/) (2024)
- [A Data-Driven Real-Time Optimal Power Flow Algorithm Using Local Feedback](https://mlsanity.com/paper/2502.15306/) (2025)
- [Wide-Area Feedback Control for Renewables-Heavy Power Systems: A Comparative Study of Reinforcement Learning and Lyapunov-Based Design](https://mlsanity.com/paper/2511.12911/) (2025)

## Abstract (arXiv)

As more inverter-connected renewable resources are integrated into the grid, frequency stability may degrade because of the reduction in mechanical inertia and damping. A common approach to mitigate this degradation in performance is to use the power electronic interfaces of the renewable resources for primary frequency control. Since inverter-connected resources can realize almost arbitrary responses to frequency changes, they are not limited to reproducing the linear droop behaviors. To fully leverage their capabilities, reinforcement learning (RL) has emerged as a popular method to design nonlinear controllers to optimize a host of objective functions. Because both inverter-connected resources and synchronous generators would be a significant part of the grid in the near and intermediate future, the learned controller of the former should be stabilizing with respect to the nonlinear dynamics of the latter. To overcome this challenge, we explicitly engineer the structure of neural network-based controllers such that they guarantee system stability by construction, through the use of a Lyapunov function. A recurrent neural network architecture is used to efficiently train the controllers. The resulting controllers only use local information and outperform optimal linear droop as well as other state-of-the-art learning approaches.