# Deep Reinforcement Learning for Community Battery Scheduling under Uncertainties of Load, PV Generation, and Energy Prices

arXiv: https://arxiv.org/abs/2312.03008 · HTML: https://mlsanity.com/paper/2312.03008/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 24 references that are in the map.
- First posted on arXiv 4 Dec 2023.

## Builds on (most cited first, 9 of 9)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Parameter Space Noise for Exploration](https://mlsanity.com/paper/1706.01905/) (2017)
- [Energy Storage Arbitrage in Real-Time Markets via Reinforcement Learning](https://mlsanity.com/paper/1711.03127/) (2017)
- [Smart Online Charging Algorithm for Electric Vehicles via Customized Actor-Critic Learning](https://mlsanity.com/paper/2106.00854/) (2021)
- [A Learning-based Optimal Market Bidding Strategy for Price-Maker Energy Storage](https://mlsanity.com/paper/2106.02396/) (2021)
- [MARL for Decentralized Electric Vehicle Charging Coordination with V2V Energy Exchange](https://mlsanity.com/paper/2308.14111/) (2023)
- [Learning a Multi-Agent Controller for Shared Energy Storage System](https://mlsanity.com/paper/2302.08328/) (2023)
- [Deep Reinforcement Learning for Wind and Energy Storage Coordination in Wholesale Energy and Ancillary Service Markets](https://mlsanity.com/paper/2212.13368/) (2022)
- [Community Battery Energy Storage Systems for Enhancing Distribution System Operation: A Multi-objective Optimization Approach](https://mlsanity.com/paper/2311.16110/) (2023)

## Abstract (arXiv)

In response to the growing uptake of distributed energy resources (DERs), community batteries have emerged as a promising solution to support renewable energy integration, reduce peak load, and enhance grid reliability. This paper presents a deep reinforcement learning (RL) strategy, centered around the soft actor-critic (SAC) algorithm, to schedule a community battery system in the presence of uncertainties, such as solar photovoltaic (PV) generation, local demand, and real-time energy prices. We position the community battery to play a versatile role, in integrating local PV energy, reducing peak load, and exploiting energy price fluctuations for arbitrage, thereby minimizing the system cost. To improve exploration and convergence during RL training, we utilize the noisy network technique. This paper conducts a comparative study of different RL algorithms, including proximal policy optimization (PPO) and deep deterministic policy gradient (DDPG) algorithms, to evaluate their effectiveness in the community battery scheduling problem. The results demonstrate the potential of RL in addressing community battery scheduling challenges and show that the SAC algorithm achieves the best performance compared to RL and optimization benchmarks.