Model Predictive Control-Guided Reinforcement Learning for Implicit Balancing
On the map: Deep reinforcement learning methods → RL for energy, networks and operations
Abstract (arXiv)
In Europe, profit-seeking balance responsible parties can deviate in real time from their day-ahead nominations to assist transmission system operators in maintaining the supply-demand balance. Model predictive control (MPC) strategies to exploit these implicit balancing strategies capture arbitrage opportunities, but fail to accurately capture the price-formation process in the European imbalance markets and face high computational costs. Model-free reinforcement learning (RL) methods are fast to execute, but require data-intensive training and usually rely on real-time and historical data for decision-making. This paper proposes an MPC-guided RL method that combines the complementary strengths of both MPC and RL. The proposed method can effectively incorporate forecasts into the decision-making process (as in MPC), while maintaining the fast inference capability of RL. The performance of the proposed method is evaluated on the implicit balancing battery control problem using Belgian balancing data from 2023. First, we analyze the performance of the standalone state-of-the-art RL and MPC methods from various angles, to highlight their individual strengths and limitations. Next, we show an arbitrage profit benefit of the proposed MPC-guided RL method of 16.15% and 54.36%, compared to standalone RL and MPC.
Key facts. Cited by 3 papers in the map (data through 13 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 18 of its 38 references that are in the map. First posted on arXiv 6 Oct 2025.
Builds on
18 of 38 references.
Closest ancestors · 1
Same direction, newest first
- 2024Control Policy Correction Framework for Reinforcement Learning-based Energy Arbitrage Strategies29 Apr
Other references · 17
- 2017A Distributional Perspective on Reinforcement Learningcited by 745
- 2017Distributional Reinforcement Learning with Quantile Regressioncited by 465
- 2019Data-driven Economic NMPC using Reinforcement Learningcited by 55
- 2021Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policiescited by 37
- 2020Blending MPC & Value Function Approximation for Efficient Reinforcement Learningcited by 29
- 2020A Deep Reinforcement Learning Framework for Continuous Intraday Market Biddingcited by 15
- 2020Learning-Based Predictive Control via Real-Time Aggregate Flexibilitycited by 7
- 2015A Data-driven Bidding Model for a Cluster of Price-responsive Consumers of Electricitycited by 6
- 2021Reinforcement Learning Based Optimal Battery Control Under Cycle-based Degradation Costcited by 5
- 2023Distributional Reinforcement Learning-based Energy Arbitrage Strategies in Imbalance Settlement Mechanismcited by 5
- 2023Demand response for residential building heating: Effective Monte Carlo Tree Search control based on physics-informed neural networkscited by 4
- 2024Probabilistic forecasting of power system imbalance using neural network-based ensemblescited by 3
5 more
- 2024Predicting and Publishing Accurate Imbalance Prices Using Monte Carlo Tree Searchcited by 3
- 2023Ensemble Nonlinear Model Predictive Control for Residential Solar-Battery Energy Managementcited by 2
- 2025Gaming Strategies in European Imbalance Settlement Mechanismscited by 2
- 2019Optimal Combination of Frequency Control and Peak Shaving with Battery Storage Systemscited by 1
- 2025Insights from Game Theory into the Impact of Smart Balancing on Power System Stabilitycited by 1
Built on by
Citing papers per year: 2026: 3.
3 papers.