mlsanityData to 7 Oct 2026

Learning the Optimal Power Flow: Environment Design Matters

arXiv 2403.17831 ↗First posted 26 Mar 2024cs.LG cs.SY eess.SYcited by 5

Not on the map. Why?

Abstract (arXiv)

To solve the optimal power flow (OPF) problem, reinforcement learning (RL) emerges as a promising new approach. However, the RL-OPF literature is strongly divided regarding the exact formulation of the OPF problem as an RL environment. In this work, we collect and implement diverse environment design decisions from the literature regarding training data, observation space, episode definition, and reward function choice. In an experimental analysis, we show the significant impact of these environment design options on RL-OPF training performance. Further, we derive some first recommendations regarding the choice of these design decisions. The created environment framework is fully open-source and can serve as a benchmark for future research in the RL-OPF field.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 15 of its 44 references that are in the map. First posted on arXiv 26 Mar 2024.

Builds on

15 of 44 references.

References · 15

  1. 2015Continuous control with deep reinforcement learningcited by 3,279
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  3. 2017Deep Reinforcement Learning that Matterscited by 727
  4. 2019Google Research Football: A Novel Reinforcement Learning Environmentcited by 257
  5. 2021Representation Matters: Offline Pretraining for Sequential Decision Makingcited by 90
  6. 2019Learning Optimal Solutions for Extremely Fast AC Optimal Power Flowcited by 76
  7. 2020Action Space Shaping in Deep Reinforcement Learningcited by 45
  8. 2020DeepOPF: A Feasibility-Optimized Deep Neural Network Approach for AC Optimal Power Flow Problemscited by 45
  9. 2016Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?cited by 44
  10. 2020Learning to Locomote: Understanding How Environment Design Matters for Deep Reinforcement Learningcited by 27
  11. 2021Gym-ANM: Reinforcement Learning Environments for Active Network Management Tasks in Electricity Distribution Systemscited by 18
  12. 2020Observation Space Matters: Benchmark and Optimization Algorithmcited by 8
3 more
  1. 2022Stability Constrained Reinforcement Learning for Decentralized Real-Time Voltage Controlcited by 8
  2. 2022Safe reinforcement learning for multi-energy management systems with known constraint functionscited by 5
  3. 2023Approximating Energy Market Clearing and Bidding With Model-Based Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 2.

5 papers.

  1. 2024Physics-Informed Graph Neural Networks for Robust AC-Optimal Power Flowcited by 1
  2. 2026Learning to Run Power Networks: Effective AlphaZero-inspired Topological Controlcited by 1
  3. 2024Adaptive Informed Deep Neural Networks for Power Flow Analysiscited by 0
  4. 2025A General Approach of Automated Environment Design for Learning the Optimal Power Flowcited by 0
  5. 2026OpenG2G: A Simulation Platform for AI Datacenter-Grid Runtime Coordinationcited by 0