# Reinforcement learning for Energies of the future and carbon neutrality: a Challenge Design

arXiv: https://arxiv.org/abs/2207.10330 · HTML: https://mlsanity.com/paper/2207.10330/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 36 references that are in the map.
- First posted on arXiv 21 Jul 2022.

## Builds on (most cited first, 13 of 13)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [Learning to run a Power Network Challenge: a Retrospective Analysis](https://mlsanity.com/paper/2103.03104/) (2021)
- [Learning to run a power network challenge for training topology controllers](https://mlsanity.com/paper/1912.04211/) (2019)
- [Action Set Based Policy Optimization for Safe Power Grid Management](https://mlsanity.com/paper/2106.15200/) (2021)
- [AI-Based Autonomous Line Flow Control via Topology Adjustment for Maximizing Time-Series ATCs](https://mlsanity.com/paper/1911.04263/) (2019)
- [Efficient and Robust Reinforcement Learning with Uncertainty-based Value Expansion](https://mlsanity.com/paper/1912.05328/) (2019)
- [An Efficient Asynchronous Method for Integrating Evolutionary and Gradient-based Policy Search](https://mlsanity.com/paper/2012.05417/) (2020)
- [Recent Advances in Deep Learning: An Overview](https://mlsanity.com/paper/1807.08169/) (2018)

## Built on by (most cited first, 8 of 8)

- [Hierarchical Reinforcement Learning for Power Network Topology Control](https://mlsanity.com/paper/2311.02129/) (2023)
- [HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach](https://mlsanity.com/paper/2405.00629/) (2024)
- [Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control](https://mlsanity.com/paper/2502.08681/) (2025)
- [Imitation Learning for Intra-Day Power Grid Operation through Topology Actions](https://mlsanity.com/paper/2407.19865/) (2024)
- [State and Action Factorization in Power Grids](https://mlsanity.com/paper/2409.04467/) (2024)
- [Fault Detection for agents on power grid topology optimization: A Comprehensive analysis](https://mlsanity.com/paper/2406.16426/) (2024)
- [Learning Topology Actions for Power Grid Control: A Graph-Based Soft-Label Imitation Learning Approach](https://mlsanity.com/paper/2503.15190/) (2025)
- [Power Grid Control with Graph-Based Distributed Reinforcement Learning](https://mlsanity.com/paper/2509.02861/) (2025)

## Abstract (arXiv)

Current rapid changes in climate increase the urgency to change energy production and consumption management, to reduce carbon and other green-house gas production. In this context, the French electricity network management company RTE (R'eseau de Transport d''Electricit'e) has recently published the results of an extensive study outlining various scenarios for tomorrow's French power management. We propose a challenge that will test the viability of such a scenario. The goal is to control electricity transportation in power networks, while pursuing multiple objectives: balancing production and consumption, minimizing energetic losses, and keeping people and equipment safe and particularly avoiding catastrophic failures. While the importance of the application provides a goal in itself, this challenge also aims to push the state-of-the-art in a branch of Artificial Intelligence (AI) called Reinforcement Learning (RL), which offers new possibilities to tackle control problems. In particular, various aspects of the combination of Deep Learning and RL called Deep Reinforcement Learning remain to be harnessed in this application domain. This challenge belongs to a series started in 2019 under the name "Learning to run a power network" (L2RPN). In this new edition, we introduce new more realistic scenarios proposed by RTE to reach carbon neutrality by 2050, retiring fossil fuel electricity production, increasing proportions of renewable and nuclear energy and introducing batteries. Furthermore, we provide a baseline using state-of-the-art reinforcement learning algorithm to stimulate the future participants.