# Arbitrage of Energy Storage in Electricity Markets with Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1904.12232 · HTML: https://mlsanity.com/paper/1904.12232/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2019.

## Built on by (most cited first, 3 of 3)

- [Proximal Policy Optimization Based Reinforcement Learning for Joint Bidding in Energy and Frequency Regulation Markets](https://mlsanity.com/paper/2212.06551/) (2022)
- [Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets](https://mlsanity.com/paper/2402.19110/) (2024)
- [State Representation Matters in Deep Reinforcement Learning: Application to Energy Trading](https://mlsanity.com/paper/2606.27032/) (2026)

## Abstract (arXiv)

In this letter, we address the problem of controlling energy storage systems (ESSs) for arbitrage in real-time electricity markets under price uncertainty. We first formulate this problem as a Markov decision process, and then develop a deep reinforcement learning based algorithm to learn a stochastic control policy that maps a set of available information processed by a recurrent neural network to ESSs' charging/discharging actions. Finally, we verify the effectiveness of our algorithm using real-time electricity prices from PJM.