mlsanityData to 14 Oct 2026

Enhancing Battery Storage Energy Arbitrage with Deep Reinforcement Learning and Time-Series Forecasting

arXiv 2410.20005 ↗First posted 25 Oct 2024cs.LG cs.AI cs.OScited by 2

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

Energy arbitrage is one of the most profitable sources of income for battery operators, generating revenues by buying and selling electricity at different prices. Forecasting these revenues is challenging due to the inherent uncertainty of electricity prices. Deep reinforcement learning (DRL) emerged in recent years as a promising tool, able to cope with uncertainty by training on large quantities of historical data. However, without access to future electricity prices, DRL agents can only react to the currently observed price and not learn to plan battery dispatch. Therefore, in this study, we combine DRL with time-series forecasting methods from deep learning to enhance the performance on energy arbitrage. We conduct a case study using price data from Alberta, Canada that is characterized by irregular price spikes and highly non-stationary. This data is challenging to forecast even when state-of-the-art deep learning models consisting of convolutional layers, recurrent layers, and attention modules are deployed. Our results show that energy arbitrage with DRL-enabled battery control still significantly benefits from these imperfect predictions, but only if predictors for several horizons are combined. Grouping multiple predictions for the next 24-hour window, accumulated rewards increased by 60% for deep Q-networks (DQN) compared to the experiments without forecasts. We hypothesize that multiple predictors, despite their imperfections, convey useful information regarding the future development of electricity prices through a "majority vote" principle, enabling the DRL agent to learn more profitable control policies.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 4 of its 42 references that are in the map. First posted on arXiv 25 Oct 2024.

Builds on

4 of 42 references.

References · 4

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2017Energy Storage Arbitrage in Real-Time Markets via Reinforcement Learningcited by 14
  4. 2021Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025The Economic Dispatch of Power-to-Gas Systems with Deep Reinforcement Learning:Tackling the Challenge of Delayed Rewards with Long-Term Energy Storagecited by 0
  2. 2026State Representation Matters in Deep Reinforcement Learning: Application to Energy Tradingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets
  2. 2024Attentive Convolutional Deep Reinforcement Learning for Optimizing Solar-Storage Systems in Real-Time Electricity Markets
  3. 2025Economic Battery Storage Dispatch with Deep Reinforcement Learning from Rule-Based Demonstrations
  4. 2024Learning a local trading strategy: deep reinforcement learning for grid-scale renewable energy integration
  5. 2026Sizing of Battery Considering Renewable Energy Bidding Strategy with Reinforcement Learning
  6. 2026Explainable Data-driven Deep Reinforcement Learning Methods for Optimal Energy Management in Buildings
  7. 2025Natural-gas storage modelling by deep reinforcement learning
  8. 2025Deep Reinforcement Learning-Based Optimization of Second-Life Battery Utilization in Electric Vehicles Charging Stations