mlsanityData to 14 Oct 2026

State Representation Matters in Deep Reinforcement Learning: Application to Energy Trading

arXiv 2606.27032 ↗First posted 25 Jun 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

Energy trading decisions depend not only on current market prices, but also on expected future market conditions, and operational constraints. This makes the state representation given to a reinforcement learning agent an important design choice. We study this in HydroDam, a pumped-storage arbitrage environment, using a fixed Double DQN agent. The environment, action space, reward function, network, and training protocol are kept fixed; only the market features are changed. We compare absolute price/calendar features, relative features that compare current prices with recent market history, forecast features, and all combinations of these three feature families. Policies are trained and selected using 2007--2011 Belgian day-ahead prices and evaluated on two test settings: a later same-market test set from 2012--2025 and 39 other ENTSO-E market zones. Absolute features only reaches 28.8% on the test set and a median 5.7% across zones. Relative-only and forecast-only states also stay below a rolling price-score heuristic in the cross-zone median. Combining feature families is much stronger: absolute + relative reaches 49.9% on the test set and a 39.8% cross-zone median, while absolute + relative + forecast reaches 55.6% and 47.5%. These results suggest that state representation is not a minor preprocessing choice in storage-trading RL, but a central part of the policy design: robust transfer requires combining price scale, recent relative price context, and short-horizon forecast information, rather than relying on any single feature family.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 8 of its 17 references that are in the map. First posted on arXiv 25 Jun 2026.

Builds on

8 of 17 references.

References · 8

  1. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  2. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  3. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170
  4. 2018State Representation Learning for Control: An Overviewcited by 127
  5. 2020Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmarkcited by 108
  6. 2017Energy Storage Arbitrage in Real-Time Markets via Reinforcement Learningcited by 14
  7. 2019Arbitrage of Energy Storage in Electricity Markets with Deep Reinforcement Learningcited by 3
  8. 2024Enhancing Battery Storage Energy Arbitrage with Deep Reinforcement Learning and Time-Series Forecastingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Natural-gas storage modelling by deep reinforcement learning
  2. 2024Temporal-Aware Deep Reinforcement Learning for Energy Storage Bidding in Energy and Contingency Reserve Markets
  3. 2024Reinforcement Learning for Bidding Strategy Optimization in Day-Ahead Energy Market
  4. 2025Dueling Deep Reinforcement Learning for Financial Time Series
  5. 2025Deep Reinforcement Learning for Automated Stock Trading: An Ensemble Strategy
  6. 2025Deep reinforcement learning for optimal trading with partial information
  7. 2025Finding Optimal Trading History in Reinforcement Learning for Stock Market Trading
  8. 2024Deep Reinforcement Learning Strategies in Finance: Insights into Asset Holding, Trading Behavior, and Purchase Diversity