mlsanityData to 14 Oct 2026

Should we use model-free or model-based control? A case study of battery management systems

arXiv 2407.15313 ↗First posted 22 Jul 2024eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

Reinforcement learning (RL) and model predictive control (MPC) each offer distinct advantages and limitations when applied to control problems in power and energy systems. Despite various studies on these methods, benchmarks remain lacking and the preference for RL over traditional controls is not well understood. In this work, we put forth a comparative analysis using RL- and MPC-based controllers for optimizing a battery management system (BMS). The BMS problem aims to minimize costs while adhering to operational limits. by adjusting the battery (dis)charging in response to fluctuating electricity prices over a time horizon. The MPC controller uses a learningbased forecast of future demand and price changes to formulate a multi-period linear program, that can be solved using off-the-shelf solvers. Meanwhile, the RL controller requires no timeseries modeling but instead is trained from the sample trajectories using the proximal policy optimization (PPO) algorithm. Numerical tests compare these controllers across optimality, training time, testing time, and robustness, providing a comprehensive evaluation of their efficacy. RL not only yields optimal solutions quickly but also ensures robustness to shifts in customer behavior, such as changes in demand distribution. However, as expected, training the RL agent is more time-consuming than MPC.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 10 of its 15 references that are in the map. First posted on arXiv 22 Jul 2024.

Builds on

10 of 15 references.

References · 10

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2015Trust Region Policy Optimizationcited by 2,580
  4. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  5. 2017Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Controlcited by 53
  6. 2019Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Controlcited by 22
  7. 2022Stability Constrained Reinforcement Learning for Decentralized Real-Time Voltage Controlcited by 9
  8. 2021Reinforcement Learning Based Optimal Battery Control Under Cycle-based Degradation Costcited by 6
  9. 2021SAVER: Safe Learning-Based Controller for Real-Time Voltage Regulationcited by 5
  10. 2019Energy Storage Management via Deep Q-Networkscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Comparing Traditional and Reinforcement-Learning Methods for Energy Storage Control
  2. 2025Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classification
  3. 2025A Reinforcement Learning Approach for Optimal Control in Microgrids
  4. 2026Learning Control Policies from Heterogeneous Multi-Horizon Time Series in Battery Energy Management Systems
  5. 2025Hierarchical RL-MPC for Demand Response Scheduling
  6. 2024Learning a local trading strategy: deep reinforcement learning for grid-scale renewable energy integration
  7. 2018Definition and evaluation of model-free coordination of electrical vehicle charging with reinforcement learning
  8. 2025Model Predictive Control-Guided Reinforcement Learning for Implicit Balancing