# Wide-Area Feedback Control for Renewables-Heavy Power Systems: A Comparative Study of Reinforcement Learning and Lyapunov-Based Design

arXiv: https://arxiv.org/abs/2511.12911 · HTML: https://mlsanity.com/paper/2511.12911/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 13 of its 50 references that are in the map.
- First posted on arXiv 17 Nov 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 13 of 13)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [Stability Constrained Reinforcement Learning for Real-Time Voltage Control](https://mlsanity.com/paper/2109.14854/) (2021)
- [Reinforcement Learning for Optimal Primary Frequency Control: A Lyapunov Approach](https://mlsanity.com/paper/2009.05654/) (2020)
- [Reinforcement Learning for Distributed Transient Frequency Control with Stability and Safety Guarantees](https://mlsanity.com/paper/2207.03329/) (2022)
- [Online Event-Triggered Switching for Frequency Control in Power Grids with Variable Inertia](https://mlsanity.com/paper/2408.15436/) (2024)
- [Dynamic Modeling, Stability, and Control of Power Systems with Distributed Energy Resources](https://mlsanity.com/paper/1804.04933/) (2018)
- [Observers for Differential Algebraic Equation Models of Power Networks: Jointly Estimating Dynamic and Algebraic States](https://mlsanity.com/paper/2202.13254/) (2022)
- [Dynamic State Estimation of Nonlinear Differential Algebraic Equation Models of Power Networks](https://mlsanity.com/paper/2206.07623/) (2022)
- [Robust Control for Renewable-Integrated Power Networks Considering Input Bound Constraints and Worst-Case Uncertainty Measure](https://mlsanity.com/paper/1802.09071/) (2018)
- [Robust Dynamic State Estimation of Synchronous Machines with Asymptotic State Estimation Error Performance Guarantees](https://mlsanity.com/paper/1910.09487/) (2019)
- [Load and Renewable-Following Control of Linearization-Free Differential Algebraic Equation Power System Models](https://mlsanity.com/paper/2104.05957/) (2021)
- [Robust Dynamic State Estimation of Multi-Machine Power Networks with Solar Farms and Dynamics Loads](https://mlsanity.com/paper/2209.10032/) (2022)

## Abstract (arXiv)

As renewable energy sources become more prevalent, accurately modeling power grid dynamics is becoming increasingly more complex. Concurrently, data acquisition and realtime system state monitoring are becoming more available for control centers. This motivates shifting from textitmodel- and Lyapunov-based feedback controller designs toward textitmodel-free ones. Reinforcement learning (RL) has emerged as a key tool for designing model-free controllers. Various studies have been carried out to study voltage/frequency control strategies via RL. However, usually a simplified system model is used neglecting detailed dynamics of solar, wind, and composite loads -- and damping system-wide oscillations and modeling power flows are all usually ignored. To that end, we pose an optimal feedback control problem for a detailed renewables-heavy power system, defined by a set of nonlinear differential algebraic equations (NDAE). The control problem is solved using a completely model-free design via RL as well as using a model-based approach built upon the Lyapunov stability theory with guarantees. The paper in its essence seeks to explore whether data-driven feedback control should be used in power grids over its model-driven counterpart. Theoretical developments and thorough case studies are presented with an eye on this exploration. Finally, a detailed analysis is provided to delineate the strengths and weaknesses of both approaches for renewables-heavy grids.