# Model Predictive Control-Guided Reinforcement Learning for Implicit Balancing

arXiv: https://arxiv.org/abs/2510.04868 · HTML: https://mlsanity.com/paper/2510.04868/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 38 references that are in the map.
- First posted on arXiv 6 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 18 of 18)

- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [Distributional Reinforcement Learning with Quantile Regression](https://mlsanity.com/paper/1710.10044/) (2017)
- [Data-driven Economic NMPC using Reinforcement Learning](https://mlsanity.com/paper/1904.04152/) (2019)
- [Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policies](https://mlsanity.com/paper/2111.02552/) (2021)
- [Blending MPC & Value Function Approximation for Efficient Reinforcement Learning](https://mlsanity.com/paper/2012.05909/) (2020)
- [A Deep Reinforcement Learning Framework for Continuous Intraday Market Bidding](https://mlsanity.com/paper/2004.05940/) (2020)
- [Learning-Based Predictive Control via Real-Time Aggregate Flexibility](https://mlsanity.com/paper/2012.11261/) (2020)
- [A Data-driven Bidding Model for a Cluster of Price-responsive Consumers of Electricity](https://mlsanity.com/paper/1506.06587/) (2015)
- [Reinforcement Learning Based Optimal Battery Control Under Cycle-based Degradation Cost](https://mlsanity.com/paper/2108.02374/) (2021)
- [Distributional Reinforcement Learning-based Energy Arbitrage Strategies in Imbalance Settlement Mechanism](https://mlsanity.com/paper/2401.00015/) (2023)
- [Demand response for residential building heating: Effective Monte Carlo Tree Search control based on physics-informed neural networks](https://mlsanity.com/paper/2312.03365/) (2023)
- [Probabilistic forecasting of power system imbalance using neural network-based ensembles](https://mlsanity.com/paper/2404.14836/) (2024)
- [Predicting and Publishing Accurate Imbalance Prices Using Monte Carlo Tree Search](https://mlsanity.com/paper/2411.04011/) (2024)
- [Ensemble Nonlinear Model Predictive Control for Residential Solar-Battery Energy Management](https://mlsanity.com/paper/2303.10393/) (2023)
- [Gaming Strategies in European Imbalance Settlement Mechanisms](https://mlsanity.com/paper/2505.14133/) (2025)
- [Optimal Combination of Frequency Control and Peak Shaving with Battery Storage Systems](https://mlsanity.com/paper/1906.06907/) (2019)
- [Control Policy Correction Framework for Reinforcement Learning-based Energy Arbitrage Strategies](https://mlsanity.com/paper/2404.18821/) (2024)
- [Insights from Game Theory into the Impact of Smart Balancing on Power System Stability](https://mlsanity.com/paper/2503.20706/) (2025)

## Built on by (most cited first, 3 of 3)

- [Neural Network-Assisted Model Predictive Control for Implicit Balancing](https://mlsanity.com/paper/2604.01805/) (2026)
- [S3TS: Stochastic Scenario-Structured Tree Search for Advanced Planning Under Uncertainty](https://mlsanity.com/paper/2606.02151/) (2026)
- [Control-Oriented Scenario Tree Construction through Reinforcement Learning](https://mlsanity.com/paper/2608.09335/) (2026)

## Abstract (arXiv)

In Europe, profit-seeking balance responsible parties can deviate in real time from their day-ahead nominations to assist transmission system operators in maintaining the supply-demand balance. Model predictive control (MPC) strategies to exploit these implicit balancing strategies capture arbitrage opportunities, but fail to accurately capture the price-formation process in the European imbalance markets and face high computational costs. Model-free reinforcement learning (RL) methods are fast to execute, but require data-intensive training and usually rely on real-time and historical data for decision-making. This paper proposes an MPC-guided RL method that combines the complementary strengths of both MPC and RL. The proposed method can effectively incorporate forecasts into the decision-making process (as in MPC), while maintaining the fast inference capability of RL. The performance of the proposed method is evaluated on the implicit balancing battery control problem using Belgian balancing data from 2023. First, we analyze the performance of the standalone state-of-the-art RL and MPC methods from various angles, to highlight their individual strengths and limitations. Next, we show an arbitrage profit benefit of the proposed MPC-guided RL method of 16.15% and 54.36%, compared to standalone RL and MPC.