# Two-stage Deep Reinforcement Learning for Inverter-based Volt-VAR Control in Active Distribution Networks

arXiv: https://arxiv.org/abs/2005.11142 · HTML: https://mlsanity.com/paper/2005.11142/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 May 2020.

## Built on by (most cited first, 12 of 12)

- [Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges](https://mlsanity.com/paper/2102.01168/) (2021)
- [A Review of Safe Reinforcement Learning Methods for Modern Power Systems](https://mlsanity.com/paper/2407.00304/) (2024)
- [Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training](https://mlsanity.com/paper/2110.08956/) (2021)
- [Networked Multiagent Safe Reinforcement Learning for Low-carbon Demand Management in Distribution Network](https://mlsanity.com/paper/2311.15594/) (2023)
- [Bi-level Off-policy Reinforcement Learning for Volt/VAR Control Involving Continuous and Discrete Devices](https://mlsanity.com/paper/2104.05902/) (2021)
- [Reducing Learning Difficulties: One-Step Two-Critic Deep Reinforcement Learning for Inverter-based Volt-Var Control](https://mlsanity.com/paper/2203.16289/) (2022)
- [Deep Reinforcement Learning for Voltage Control and Renewable Accommodation Using Spatial-Temporal Graph Information](https://mlsanity.com/paper/2401.15848/) (2024)
- [Robust Deep Reinforcement Learning for Inverter-based Volt-Var Control in Partially Observable Distribution Networks](https://mlsanity.com/paper/2408.06776/) (2024)
- [Residual Deep Reinforcement Learning for Inverter-based Volt-Var Control](https://mlsanity.com/paper/2408.06790/) (2024)
- [Two-Stage Active Distribution Network Voltage Control via LLM-RL Collaboration: A Hybrid Knowledge-Data-Driven Approach](https://mlsanity.com/paper/2602.21715/) (2026)
- [Reducing Action Space: Reference-Model-Assisted Deep Reinforcement Learning for Inverter-based Volt-Var Control](https://mlsanity.com/paper/2210.07360/) (2022)
- [Augmented Lagrangian-Based Safe Reinforcement Learning Approach for Distribution System Volt/VAR Control](https://mlsanity.com/paper/2410.15188/) (2024)

## Abstract (arXiv)

Model-based Vol/VAR optimization method is widely used to eliminate voltage violations and reduce network losses. However, the parameters of active distribution networks(ADNs) are not onsite identified, so significant errors may be involved in the model and make the model-based method infeasible. To cope with this critical issue, we propose a novel two-stage deep reinforcement learning (DRL) method to improve the voltage profile by regulating inverter-based energy resources, which consists of offline stage and online stage. In the offline stage, a highly efficient adversarial reinforcement learning algorithm is developed to train an offline agent robust to the model mismatch. In the sequential online stage, we transfer the offline agent safely as the online agent to perform continuous learning and controlling online with significantly improved safety and efficiency. Numerical simulations on IEEE test cases not only demonstrate that the proposed adversarial reinforcement learning algorithm outperforms the state-of-art algorithm, but also show that our proposed two-stage method achieves much better performance than the existing DRL based methods in the online application.