# Deep Reinforcement Learning for Optimal Power Flow with Renewables Using Graph Information

arXiv: https://arxiv.org/abs/2112.11461 · HTML: https://mlsanity.com/paper/2112.11461/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 64 references that are in the map.
- First posted on arXiv 22 Dec 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs](https://mlsanity.com/paper/1704.02901/) (2017)
- [Sample Efficient Actor-Critic with Experience Replay](https://mlsanity.com/paper/1611.01224/) (2016)
- [Predicting AC Optimal Power Flows: Combining Deep Learning and Lagrangian Dual Methods](https://mlsanity.com/paper/1909.10461/) (2019)
- [Local Spectral Graph Convolution for Point Set Feature Learning](https://mlsanity.com/paper/1803.05827/) (2018)
- [DeepOPF: Deep Neural Network for DC Optimal Power Flow](https://mlsanity.com/paper/1905.04479/) (2019)
- [End-to-end Flow Correlation Tracking with Spatial-temporal Attention](https://mlsanity.com/paper/1711.01124/) (2017)
- [Data-driven Optimal Power Flow: A Physics-Informed Machine Learning Approach](https://mlsanity.com/paper/2006.00544/) (2020)
- [Data-driven Local Control Design using Optimization and Machine Learning Techniques](https://mlsanity.com/paper/1808.01009/) (2018)
- [Data-based Distributionally Robust Stochastic Optimal Power Flow, Part I: Methodologies](https://mlsanity.com/paper/1804.06388/) (2018)
- [Information Freshness-Aware Task Offloading in Air-Ground Integrated Edge Computing Systems](https://mlsanity.com/paper/2007.10129/) (2020)
- [A Robust Approach to Chance Constrained Optimal Power Flow with Renewable Generation](https://mlsanity.com/paper/1504.06011/) (2015)
- [Stochastic Optimal Power Flow Based on Data-Driven Distributionally Robust Optimization](https://mlsanity.com/paper/1706.04267/) (2017)

## Built on by (most cited first, 1 of 1)

- [Machine Learning for Electricity Market Clearing](https://mlsanity.com/paper/2205.11641/) (2022)

## Abstract (arXiv)

Renewable energy resources (RERs) have been increasingly integrated into large-scale distributed power systems. Considering uncertainties and voltage fluctuation issues introduced by RERs, in this paper, we propose a deep reinforcement learning (DRL)-based strategy leveraging spatial-temporal (ST) graphical information of power systems, to dynamically search for the optimal operation, i.e., optimal power flow (OPF), of power systems with a high uptake of RERs. Specifically, we formulate the OPF problem as a multi-objective optimization problem considering generation cost, voltage fluctuation, and transmission loss, and employ deep deterministic policy gradient (DDPG) to learn an optimal allocation strategy for OPF. Moreover, given that the nodes in power systems are self-correlated and interrelated in temporal and spatial views, we develop a multi-grained attention-based spatial-temporal graph convolution network (MG-ASTGCN) for extracting ST graphical correlations and features, aiming to provide prior knowledge of power systems for its sequential DDPG algorithm to more effectively solve OPF. We validate our algorithm on modified IEEE 33, 69, and 118-bus radial distribution systems and demonstrate that our algorithm outperforms other benchmark algorithms. Our experimental results also reveal that our MG-ASTGCN can significantly accelerate DDPG's training process and performance in solving OPF.