Safe Imitation Learning-based Optimal Energy Storage Systems Dispatch in Distribution Networks
On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling
Abstract (arXiv)
The integration of distributed energy resources (DER) has escalated the challenge of voltage magnitude regulation in distribution networks. Traditional model-based approaches, which rely on complex sequential mathematical formulations, struggle to meet real-time operational demands. Deep reinforcement learning (DRL) offers a promising alternative by enabling offline training with distribution network simulators, followed by real-time execution. However, DRL algorithms tend to converge to local optima due to limited exploration efficiency. Additionally, DRL algorithms can not enforce voltage magnitude constraints, leading to potential operational violations when implemented in the distribution network operation. This study addresses these challenges by proposing a novel safe imitation reinforcement learning (IRL) framework that combines IRL and a designed safety layer, aiming to optimize the operation of Energy Storage Systems (ESSs) in active distribution networks. The proposed safe IRL framework comprises two phases: offline training and online execution. During the offline phase, optimal state-action pairs are collected using an NLP solver, guiding the IRL policy iteration. In the online phase, the trained IRL policy's decisions are adjusted by the safety layer to maintain safety and constraint compliance. Simulation results demonstrate the efficacy of Safe IRL in balancing operational efficiency and safety, eliminating voltage violations, and maintaining low operation cost errors across various network sizes, while meeting real-time execution requirements.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 9 of its 22 references that are in the map. First posted on arXiv 1 Nov 2024.
Builds on
9 of 22 references.
References · 9
- 2015Continuous control with deep reinforcement learningcited by 3,407
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
- 2021Imitation Learning: Progress, Taxonomies and Challengescited by 58
- 2021Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networkscited by 54
- 2023Optimal Energy System Scheduling Using A Constraint-Aware Reinforcement Learning Algorithmcited by 11
- 2022Performance Comparison of Deep RL Algorithms for Energy Systems Optimal Schedulingcited by 4
- 2024RL-ADN: A High-Performance Deep Reinforcement Learning Environment for Optimal Energy Storage Systems Dispatch in Active Distribution Networkscited by 4
- 2020MILP-based Imitation Learning for HVAC controlcited by 2
- 2023A Constraint Enforcement Deep Reinforcement Learning Framework for Optimal Energy Storage Systems Dispatchcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024A Critical Review of Safe Reinforcement Learning Techniques in Smart Grid Applications
- 2024Robust Deep Reinforcement Learning for Volt-VAR Optimization in Active Distribution System under Uncertainty
- 2021Reinforcement Learning for Battery Energy Storage Dispatch augmented with Model-based Optimizer
- 2024Implementing Deep Reinforcement Learning-Based Grid Voltage Control in Real-World Power Systems: Challenges and Insights
- 2024Safe Reinforcement Learning for Power System Control: A Review
- 2026A Human-AI Teaming Framework for Deep Reinforcement Learning-Based Voltage Regulation in Distribution Networks
- 2020Rethink AI-based Power Grid Control: Diving Into Algorithm Design
- 2022Efficient Learning of Voltage Control Strategies via Model-based Deep Reinforcement Learning