# Safe Imitation Learning-based Optimal Energy Storage Systems Dispatch in Distribution Networks

arXiv: https://arxiv.org/abs/2411.00995 · HTML: https://mlsanity.com/paper/2411.00995/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 9 of its 22 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Placed in

- Offline RL and imitation learning from data / [Decision transformers and sequence modeling](https://mlsanity.com/offlinerl/decision-transformers-and-sequence-modeling/)

## Builds on (most cited first, 9 of 9)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Imitation Learning: Progress, Taxonomies and Challenges](https://mlsanity.com/paper/2106.12177/) (2021)
- [Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks](https://mlsanity.com/paper/2110.14300/) (2021)
- [Optimal Energy System Scheduling Using A Constraint-Aware Reinforcement Learning Algorithm](https://mlsanity.com/paper/2305.05484/) (2023)
- [Performance Comparison of Deep RL Algorithms for Energy Systems Optimal Scheduling](https://mlsanity.com/paper/2208.00728/) (2022)
- [RL-ADN: A High-Performance Deep Reinforcement Learning Environment for Optimal Energy Storage Systems Dispatch in Active Distribution Networks](https://mlsanity.com/paper/2408.03685/) (2024)
- [MILP-based Imitation Learning for HVAC control](https://mlsanity.com/paper/2012.00286/) (2020)
- [A Constraint Enforcement Deep Reinforcement Learning Framework for Optimal Energy Storage Systems Dispatch](https://mlsanity.com/paper/2307.14304/) (2023)

## Abstract (arXiv)

The integration of distributed energy resources (DER) has escalated the challenge of voltage magnitude regulation in distribution networks. Traditional model-based approaches, which rely on complex sequential mathematical formulations, struggle to meet real-time operational demands. Deep reinforcement learning (DRL) offers a promising alternative by enabling offline training with distribution network simulators, followed by real-time execution. However, DRL algorithms tend to converge to local optima due to limited exploration efficiency. Additionally, DRL algorithms can not enforce voltage magnitude constraints, leading to potential operational violations when implemented in the distribution network operation. This study addresses these challenges by proposing a novel safe imitation reinforcement learning (IRL) framework that combines IRL and a designed safety layer, aiming to optimize the operation of Energy Storage Systems (ESSs) in active distribution networks. The proposed safe IRL framework comprises two phases: offline training and online execution. During the offline phase, optimal state-action pairs are collected using an NLP solver, guiding the IRL policy iteration. In the online phase, the trained IRL policy's decisions are adjusted by the safety layer to maintain safety and constraint compliance. Simulation results demonstrate the efficacy of Safe IRL in balancing operational efficiency and safety, eliminating voltage violations, and maintaining low operation cost errors across various network sizes, while meeting real-time execution requirements.