# Reinforcement Learning-Based Cooperative P2P Power Trading between DC Nanogrid Clusters with Wind and PV Energy Resources

arXiv: https://arxiv.org/abs/2209.07744 · HTML: https://mlsanity.com/paper/2209.07744/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 74 references that are in the map.
- First posted on arXiv 16 Sep 2022.

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Deep Recurrent Q-Learning for Partially Observable MDPs](https://mlsanity.com/paper/1507.06527/) (2015)
- [Untangling Blockchain: A Data Processing View of Blockchain Systems](https://mlsanity.com/paper/1708.05665/) (2017)
- [Path Planning of Cleaning Robot with Reinforcement Learning](https://mlsanity.com/paper/2208.08211/) (2022)
- [Peer-to-Peer Energy Trading with Sustainable User Participation: A Game Theoretic Approach](https://mlsanity.com/paper/1809.11034/) (2018)
- [Anomaly Detection of Smart Metering System for Power Management with Battery Storage System/Electric Vehicle](https://mlsanity.com/paper/2207.09784/) (2022)
- [A Motivational Game-Theoretic Approach for Peer-to-Peer Energy Trading in the Smart Grid](https://mlsanity.com/paper/1903.03922/) (2019)
- [Development of Charging, Discharging Scheduling Algorithm for Economical and Energy Efficient Operation of Multi EV Charging Station](https://mlsanity.com/paper/2205.04116/) (2022)
- [A Study of Energy Trading in a Low-Voltage Network: Centralised and Distributed Approaches](https://mlsanity.com/paper/1809.06974/) (2018)
- [Optimal Scheduling of Energy Storage for Power System with Capability of Sensing Short-term Future PV Power Production](https://mlsanity.com/paper/2112.03551/) (2021)

## Built on by (most cited first, 1 of 1)

- [Graph Reinforcement Learning for Power Grids: A Comprehensive Survey](https://mlsanity.com/paper/2407.04522/) (2024)

## Abstract (arXiv)

In replacing fossil fuels with renewable energy resources for carbon neutrality, the unbalanced resource production of intermittent wind and photovoltaic (PV) power is a critical issue for peer-to-peer (P2P) power trading. To address this issue, a reinforcement learning (RL) technique is introduced in this paper. For RL, a graph convolutional network (GCN) and a bi-directional long short-term memory (Bi-LSTM) network are jointly applied to P2P power trading between nanogrid clusters, based on cooperative game theory. The flexible and reliable DC nanogrid is suitable for integrating renewable energy for a distribution system. Each local nanogrid cluster takes the position of prosumer, focusing on power production and consumption simultaneously. For the power management of nanogrid cluster, multi-objective optimization is applied to each local nanogrid cluster with the Internet of Things (IoT) technology. Charging/discharging of an electric vehicle (EV) is executed considering the intermittent characteristics of wind and PV power production. RL algorithms, such as GCN- convolutional neural network (CNN) layers for deep Q-learning network (DQN), GCN-LSTM layers for deep recurrent Q-learning network (DRQN), GCN-Bi-LSTM layers for DRQN, and GCN-Bi-LSTM layers for proximal policy optimization (PPO), are used for simulations. Consequently, the cooperative P2P power trading system maximizes the profit by considering the time of use (ToU) tariff-based electricity cost and the system marginal price (SMP), and minimizes the amount of grid power consumption. Power management of nanogrid clusters with P2P power trading is simulated on a distribution test feeder in real time, and the proposed GCN-Bi-LSTM-PPO technique achieving the lowest electricity cost among the RL algorithms used for comparison reduces the electricity cost by 36.7%, averaging over nanogrid clusters.