# Reinforcement Learning for Battery Energy Storage Dispatch augmented with Model-based Optimizer

arXiv: https://arxiv.org/abs/2109.01659 · HTML: https://mlsanity.com/paper/2109.01659/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 9 of its 28 references that are in the map.
- First posted on arXiv 2 Sep 2021.

## Placed in

- Offline RL and imitation learning from data / [Decision transformers and sequence modeling](https://mlsanity.com/offlinerl/decision-transformers-and-sequence-modeling/)

## Builds on (most cited first, 9 of 9)

- [Adaptive Power System Emergency Control using Deep Reinforcement Learning](https://mlsanity.com/paper/1903.03712/) (2019)
- [Learning to Solve Network Flow Problems via Neural Decoding](https://mlsanity.com/paper/2002.04091/) (2020)
- [Accelerated Deep Reinforcement Learning Based Load Shedding for Emergency Voltage Control](https://mlsanity.com/paper/2006.12667/) (2020)
- [Optimal Battery Participation in Frequency Regulation Markets](https://mlsanity.com/paper/1710.10514/) (2017)
- [Bi-Level Volt-VAR Optimization to Coordinate Smart Inverters with Voltage Control Devices](https://mlsanity.com/paper/1912.03596/) (2019)
- [Learning to Solve AC Optimal Power Flow by Differentiating through Holomorphic Embeddings](https://mlsanity.com/paper/2012.09622/) (2020)
- [A Comparison of Policies on the Participation of Storage in U.S. Frequency Regulation Markets](https://mlsanity.com/paper/1602.04420/) (2016)
- [Optimal Distributed Feedback Voltage Control under Limited Reactive Power](https://mlsanity.com/paper/1810.11121/) (2018)
- [Strategies for Network-Safe Load Control with a Third-Party Aggregator and a Distribution Operator](https://mlsanity.com/paper/2012.02152/) (2020)

## Built on by (most cited first, 1 of 1)

- [Economic Battery Storage Dispatch with Deep Reinforcement Learning from Rule-Based Demonstrations](https://mlsanity.com/paper/2504.04326/) (2025)

## Abstract (arXiv)

Reinforcement learning has been found useful in solving optimal power flow (OPF) problems in electric power distribution systems. However, the use of largely model-free reinforcement learning algorithms that completely ignore the physics-based modeling of the power grid compromises the optimizer performance and poses scalability challenges. This paper proposes a novel approach to synergistically combine the physics-based models with learning-based algorithms using imitation learning to solve distribution-level OPF problems. Specifically, we propose imitation learning based improvements in deep reinforcement learning (DRL) methods to solve the OPF problem for a specific case of battery storage dispatch in the power distribution systems. The proposed imitation learning algorithm uses the approximate optimal solutions obtained from a linearized model-based OPF solver to provide a good initial policy for the DRL algorithms while improving the training efficiency. The effectiveness of the proposed approach is demonstrated using IEEE 34-bus and 123-bus distribution feeders with numerous distribution-level battery storage systems.