# Deep Reinforcement Learning for Voltage Control and Renewable Accommodation Using Spatial-Temporal Graph Information

arXiv: https://arxiv.org/abs/2401.15848 · HTML: https://mlsanity.com/paper/2401.15848/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 15 of its 52 references that are in the map.
- First posted on arXiv 29 Jan 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs](https://mlsanity.com/paper/1704.02901/) (2017)
- [Sample Efficient Actor-Critic with Experience Replay](https://mlsanity.com/paper/1611.01224/) (2016)
- [Two-Timescale Voltage Control in Distribution Grids Using Deep Reinforcement Learning](https://mlsanity.com/paper/1904.09374/) (2019)
- [Deep Reinforcement Learning Based Volt-VAR Optimization in Smart Distribution Systems](https://mlsanity.com/paper/2003.03681/) (2020)
- [Two-stage Deep Reinforcement Learning for Inverter-based Volt-VAR Control in Active Distribution Networks](https://mlsanity.com/paper/2005.11142/) (2020)
- [A Fully Distributed Reactive Power Optimization and Control Method for Active Distribution Networks](https://mlsanity.com/paper/1407.5032/) (2014)
- [Distributed reactive power feedback control for voltage regulation and loss minimization](https://mlsanity.com/paper/1303.7173/) (2013)
- [Model-Free Voltage Regulation of Unbalanced Distribution Network Based on Surrogate Model and Deep Reinforcement Learning](https://mlsanity.com/paper/2006.13992/) (2020)
- [Photovoltaic Inverter Controllers Seeking AC Optimal Power Flow Solutions](https://mlsanity.com/paper/1501.00188/) (2014)
- [Linear/Quadratic Programming-Based Optimal Power Flow using Linear Power Flow and Absolute Loss Approximations](https://mlsanity.com/paper/1711.00317/) (2017)

## Built on by (most cited first, 1 of 1)

- [Graph Reinforcement Learning for Power Grids: A Comprehensive Survey](https://mlsanity.com/paper/2407.04522/) (2024)

## Abstract (arXiv)

Renewable energy resources (RERs) have been increasingly integrated into distribution networks (DNs) for decarbonization. However, the variable nature of RERs introduces uncertainties to DNs, frequently resulting in voltage fluctuations that threaten system security and hamper the further adoption of RERs. To incentivize more RER penetration, we propose a deep reinforcement learning (DRL)-based strategy to dynamically balance the trade-off between voltage fluctuation control and renewable accommodation. To further extract multi-time-scale spatial-temporal (ST) graphical information of a DN, our strategy draws on a multi-grained attention-based spatial-temporal graph convolution network (MG-ASTGCN), consisting of ST attention mechanism and ST convolution to explore the node correlations in the spatial and temporal views. The continuous decision-making process of balancing such a trade-off can be modeled as a Markov decision process optimized by the deep deterministic policy gradient (DDPG) algorithm with the help of the derived ST information. We validate our strategy on the modified IEEE 33, 69, and 118-bus radial distribution systems, with outcomes significantly outperforming the optimization-based benchmarks. Simulations also reveal that our developed MG-ASTGCN can to a great extent accelerate the convergence speed of DDPG and improve its performance in stabilizing node voltage in an RER-rich DN. Moreover, our method improves the DN's robustness in the presence of generator failures.