Wide-Area Feedback Control for Renewables-Heavy Power Systems: A Comparative Study of Reinforcement Learning and Lyapunov-Based Design
On the map: Deep reinforcement learning methods → RL for energy, networks and operations
Abstract (arXiv)
As renewable energy sources become more prevalent, accurately modeling power grid dynamics is becoming increasingly more complex. Concurrently, data acquisition and realtime system state monitoring are becoming more available for control centers. This motivates shifting from textitmodel- and Lyapunov-based feedback controller designs toward textitmodel-free ones. Reinforcement learning (RL) has emerged as a key tool for designing model-free controllers. Various studies have been carried out to study voltage/frequency control strategies via RL. However, usually a simplified system model is used neglecting detailed dynamics of solar, wind, and composite loads -- and damping system-wide oscillations and modeling power flows are all usually ignored. To that end, we pose an optimal feedback control problem for a detailed renewables-heavy power system, defined by a set of nonlinear differential algebraic equations (NDAE). The control problem is solved using a completely model-free design via RL as well as using a model-based approach built upon the Lyapunov stability theory with guarantees. The paper in its essence seeks to explore whether data-driven feedback control should be used in power grids over its model-driven counterpart. Theoretical developments and thorough case studies are presented with an eye on this exploration. Finally, a detailed analysis is provided to delineate the strengths and weaknesses of both approaches for renewables-heavy grids.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 50 references that are in the map. First posted on arXiv 17 Nov 2025.
Builds on
13 of 50 references.
References · 13
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2021Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challengescited by 50
- 2021Stability Constrained Reinforcement Learning for Real-Time Voltage Controlcited by 15
- 2020Reinforcement Learning for Optimal Primary Frequency Control: A Lyapunov Approachcited by 14
- 2022Reinforcement Learning for Distributed Transient Frequency Control with Stability and Safety Guaranteescited by 4
- 2024Online Event-Triggered Switching for Frequency Control in Power Grids with Variable Inertiacited by 3
- 2018Dynamic Modeling, Stability, and Control of Power Systems with Distributed Energy Resourcescited by 2
- 2022Observers for Differential Algebraic Equation Models of Power Networks: Jointly Estimating Dynamic and Algebraic Statescited by 2
- 2022Dynamic State Estimation of Nonlinear Differential Algebraic Equation Models of Power Networkscited by 2
- 2018Robust Control for Renewable-Integrated Power Networks Considering Input Bound Constraints and Worst-Case Uncertainty Measurecited by 1
- 2019Robust Dynamic State Estimation of Synchronous Machines with Asymptotic State Estimation Error Performance Guaranteescited by 1
- 2021Load and Renewable-Following Control of Linearization-Free Differential Algebraic Equation Power System Modelscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Optimizing Power Grid Topologies with Reinforcement Learning: A Survey of Methods and Challenges
- 2024Safe Reinforcement Learning for Power System Control: A Review
- 2024A Review of Safe Reinforcement Learning Methods for Modern Power Systems
- 2024Reinforcement Learning for Efficient Design and Control Co-optimisation of Energy Systems
- 2025RL2Grid: Benchmarking Reinforcement Learning in Power Grid Operations
- 2025Partially Observable Residual Reinforcement Learning for PV-Inverter-Based Voltage Control in Distribution Grids
- 2025Leveraging Predictions in Power System Voltage Control: An Adaptive Approach ↗
- 2024Learning a local trading strategy: deep reinforcement learning for grid-scale renewable energy integration