mlsanityData to 14 Oct 2026

Augmented Lagrangian-Based Safe Reinforcement Learning Approach for Distribution System Volt/VAR Control

arXiv 2410.15188 ↗First posted 19 Oct 2024cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

This paper proposes a data-driven solution for Volt-VAR control problem in active distribution system. As distribution system models are always inaccurate and incomplete, it is quite difficult to solve the problem. To handle with this dilemma, this paper formulates the Volt-VAR control problem as a constrained Markov decision process (CMDP). By synergistically combining the augmented Lagrangian method and soft actor critic algorithm, a novel safe off-policy reinforcement learning (RL) approach is proposed in this paper to solve the CMDP. The actor network is updated in a policy gradient manner with the Lagrangian value function. A double-critics network is adopted to synchronously estimate the action-value function to avoid overestimation bias. The proposed algorithm does not require strong convexity guarantee of examined problems and is sample efficient. A two-stage strategy is adopted for offline training and online execution, so the accurate distribution system model is no longer needed. To achieve scalability, a centralized training distributed execution strategy is adopted for a multi-agent framework, which enables a decentralized Volt-VAR control for large-scale distribution system. Comprehensive numerical experiments with real-world electricity data demonstrate that our proposed algorithm can achieve high solution optimality and constraints compliance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 6 of its 20 references that are in the map. First posted on arXiv 19 Oct 2024.

Builds on

6 of 20 references.

References · 6

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  4. 2017pandapower - an Open Source Python Tool for Convenient Modeling, Analysis and Optimization of Electric Power Systemscited by 85
  5. 2020Two-stage Deep Reinforcement Learning for Inverter-based Volt-VAR Control in Active Distribution Networkscited by 12
  6. 2018Optimal Tap Setting of Voltage Regulation Transformers Using Batch Reinforcement Learningcited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Safety Constrained Multi-Agent Reinforcement Learning for Active Voltage Control
  2. 2025Distributed Reinforcement Learning using Local Smart Meter Data for Voltage Regulation in Distribution Networks
  3. 2024Robust Deep Reinforcement Learning for Volt-VAR Optimization in Active Distribution System under Uncertainty
  4. 2026Safe Decentralized Operation of EV Virtual Power Plant with Limited Network Visibility via Multi-Agent Reinforcement Learning
  5. 2024A Review of Safe Reinforcement Learning Methods for Modern Power Systems
  6. 2024Robust Deep Reinforcement Learning for Inverter-based Volt-Var Control in Partially Observable Distribution Networks
  7. 2024Reinforcement Learning Based Robust Volt/Var Control in Active Distribution Networks With Imprecisely Known Delay ↗
  8. 2026A Human-AI Teaming Framework for Deep Reinforcement Learning-Based Voltage Regulation in Distribution Networks