State and Action Factorization in Power Grids
Not on the map. Why?
Abstract (arXiv)
The increase of renewable energy generation towards the zero-emission target is making the problem of controlling power grids more and more challenging. The recent series of competitions Learning To Run a Power Network (L2RPN) have encouraged the use of Reinforcement Learning (RL) for the assistance of human dispatchers in operating power grids. All the solutions proposed so far severely restrict the action space and are based on a single agent acting on the entire grid or multiple independent agents acting at the substations level. In this work, we propose a domain-agnostic algorithm that estimates correlations between state and action components entirely based on data. Highly correlated state-action pairs are grouped together to create simpler, possibly independent subproblems that can lead to distinct learning processes with less computational and data requirements. The algorithm is validated on a power grid benchmark obtained with the Grid2Op simulator that has been used throughout the aforementioned competitions, showing that our algorithm is in line with domain-expert analysis. Based on these results, we lay a theoretically-grounded foundation for using distributed reinforcement learning in order to improve the existing solutions.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 13 of its 20 references that are in the map. First posted on arXiv 3 Sep 2024.
Builds on
13 of 20 references.
References · 13
- 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
- 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
- 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 111
- 2021Learning to run a Power Network Challenge: a Retrospective Analysiscited by 52
- 2017Estimating Mutual Information for Discrete-Continuous Mixturescited by 34
- 2022Controlling Commercial Cooling Systems Using Reinforcement Learningcited by 25
- 2020Reinforcement Learning for Electricity Network Operationcited by 20
- 2022Power Grid Congestion Management via Topology Optimization with AlphaZerocited by 20
- 2022PowRL: A Reinforcement Learning Framework for Robust Management of Power Networkscited by 17
- 2023Hierarchical Reinforcement Learning for Power Network Topology Controlcited by 11
- 2024HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approachcited by 10
- 2022Reinforcement learning for Energies of the future and carbon neutrality: a Challenge Designcited by 8
1 more
- 2017Guided Machine Learning for power grid segmentationcited by 1
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.