mlsanityData to 14 Oct 2026

The Economic Dispatch of Power-to-Gas Systems with Deep Reinforcement Learning:Tackling the Challenge of Delayed Rewards with Long-Term Energy Storage

arXiv 2506.06484 ↗First posted 6 Jun 2025eess.SY cs.AI cs.LGcited by 0

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

Power-to-Gas (P2G) technologies gain recognition for enabling the integration of intermittent renewables, such as wind and solar, into electricity grids. However, determining the most cost-effective operation of these systems is complex due to the volatile nature of renewable energy, electricity prices, and loads. Additionally, P2G systems are less efficient in converting and storing energy compared to battery energy storage systems (BESs), and the benefits of converting electricity into gas are not immediately apparent. Deep Reinforcement Learning (DRL) has shown promise in managing the operation of energy systems amidst these uncertainties. Yet, DRL techniques face difficulties with the delayed reward characteristic of P2G system operation. Previous research has mostly focused on short-term studies that look at the energy conversion process, neglecting the long-term storage capabilities of P2G. This study presents a new method by thoroughly examining how DRL can be applied to the economic operation of P2G systems, in combination with BESs and gas turbines, over extended periods. Through three progressively more complex case studies, we assess the performance of DRL algorithms, specifically Deep Q-Networks and Proximal Policy Optimization, and introduce modifications to enhance their effectiveness. These modifications include integrating forecasts, implementing penalties on the reward function, and applying strategic cost calculations, all aimed at addressing the issue of delayed rewards. Our findings indicate that while DRL initially struggles with the complex decision-making required for P2G system operation, the adjustments we propose significantly improve its capability to devise cost-effective operation strategies, thereby unlocking the potential for long-term energy storage in P2G technologies.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 5 of its 32 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

5 of 32 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Enhancing Battery Storage Energy Arbitrage with Deep Reinforcement Learning and Time-Series Forecasting25 Oct

Other references · 4

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  3. 2021Data-driven battery operation for energy arbitrage using rainbow deep reinforcement learningcited by 9
  4. 2023Optimal Economic Gas Turbine Dispatch with Deep Reinforcement Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Learning a local trading strategy: deep reinforcement learning for grid-scale renewable energy integration
  2. 2024Attentive Convolutional Deep Reinforcement Learning for Optimizing Solar-Storage Systems in Real-Time Electricity Markets
  3. 2025Economic Battery Storage Dispatch with Deep Reinforcement Learning from Rule-Based Demonstrations
  4. 2024Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets
  5. 2024A Holistic Power Optimization Approach for Microgrid Control Based on Deep Reinforcement Learning
  6. 2026Explainable Data-driven Deep Reinforcement Learning Methods for Optimal Energy Management in Buildings
  7. 2024Advancements in Electric Vehicle Charging Optimization: A Survey of Reinforcement Learning Approaches
  8. 2021Reinforcement Learning for Battery Energy Storage Dispatch augmented with Model-based Optimizer