# Decentralized Coordination of Distributed Energy Resources through Local Energy Markets and Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/2404.13142 · HTML: https://mlsanity.com/paper/2404.13142/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 51 references that are in the map.
- First posted on arXiv 19 Apr 2024.

## Builds on (most cited first, 12 of 12)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Emergent Tool Use From Multi-Agent Autocurricula](https://mlsanity.com/paper/1909.07528/) (2019)
- [What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Study](https://mlsanity.com/paper/2006.05990/) (2020)
- [Shared Experience Actor-Critic for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2006.07169/) (2020)
- [A Closer Look at Deep Policy Gradients](https://mlsanity.com/paper/1811.02553/) (2018)
- [Distributed Deep Reinforcement Learning for Intelligent Load Scheduling in Residential Smart Grids](https://mlsanity.com/paper/2006.16100/) (2020)
- [Generalization, Mayhems and Limits in Recurrent Proximal Policy Optimization](https://mlsanity.com/paper/2205.11104/) (2022)
- [MERLIN: Multi-agent offline and transfer learning for occupant-centric energy flexible operation of grid-interactive communities using smart meter data and CityLearn](https://mlsanity.com/paper/2301.01148/) (2022)
- [Peer-to-Peer Energy Systems for Connected Communities: A Review of Recent Advances and Emerging Challenges](https://mlsanity.com/paper/2011.11118/) (2020)
- [Impact of Local Energy Markets on the Distribution Systems: A Comprehensive Review](https://mlsanity.com/paper/2103.16137/) (2021)
- [Transactive Local Energy Markets Enable Community-Level Resource Coordination Using Individual Rewards](https://mlsanity.com/paper/2403.15617/) (2024)

## Abstract (arXiv)

As distributed energy resources (DERs) grow, the electricity grid faces increased net load variability at the grid edge, impacting operability and reliability. Transactive energy, facilitated through local energy markets, offers a decentralized, indirect demand response solution, with model-free control techniques, such as deep reinforcement learning (DRL), enabling automated, decentralized participation. However, existing studies largely overlook community-level net load variability, focusing instead on socioeconomic metrics. This study addresses this gap by using DRL agents to automate end-user participation in a local energy market (ALEX), where agents act independently to minimize individual energy bills. Results reveal a strong link between bill reduction and decreased net load variability, assessed across metrics such as ramping rate, load factor, and peak demand over various time horizons. Using a no-control baseline, DRL agents are benchmarked against a near-optimal dynamic programming approach. The dynamic programming benchmark achieves reductions of 22.05 percent, 83.92 percent, and 24.09 percent in daily import, export, and peak demand, respectively, while the DRL agents show comparable or superior results with reductions of 21.93 percent, 84.46 percent, and 27.02 percent. This study demonstrates the effectiveness of DRL in decentralized grid management, highlighting its scalability and near-optimal performance in reducing net load variability within community-driven energy markets.