# HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach

arXiv: https://arxiv.org/abs/2405.00629 · HTML: https://mlsanity.com/paper/2405.00629/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 1 May 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model](https://mlsanity.com/paper/1911.08265/) (2019)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://mlsanity.com/paper/1712.01815/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [BOHB: Robust and Efficient Hyperparameter Optimization at Scale](https://mlsanity.com/paper/1807.01774/) (2018)
- [Population Based Training of Neural Networks](https://mlsanity.com/paper/1711.09846/) (2017)
- [pandapower - an Open Source Python Tool for Convenient Modeling, Analysis and Optimization of Electric Power Systems](https://mlsanity.com/paper/1709.06743/) (2017)
- [Learning to run a Power Network Challenge: a Retrospective Analysis](https://mlsanity.com/paper/2103.03104/) (2021)
- [Learning to run a power network challenge for training topology controllers](https://mlsanity.com/paper/1912.04211/) (2019)
- [Action Set Based Policy Optimization for Safe Power Grid Management](https://mlsanity.com/paper/2106.15200/) (2021)
- [Reinforcement Learning for Electricity Network Operation](https://mlsanity.com/paper/2003.07339/) (2020)
- [Power Grid Congestion Management via Topology Optimization with AlphaZero](https://mlsanity.com/paper/2211.05612/) (2022)
- [PowRL: A Reinforcement Learning Framework for Robust Management of Power Networks](https://mlsanity.com/paper/2212.02397/) (2022)
- [Managing power grids through topology actions: A comparative study between advanced rule-based and reinforcement learning agents](https://mlsanity.com/paper/2304.00765/) (2023)
- [AI-Based Autonomous Line Flow Control via Topology Adjustment for Maximizing Time-Series ATCs](https://mlsanity.com/paper/1911.04263/) (2019)
- [Exploring grid topology reconfiguration using a simple deep reinforcement learning approach](https://mlsanity.com/paper/2011.13465/) (2020)
- [Hierarchical Reinforcement Learning for Power Network Topology Control](https://mlsanity.com/paper/2311.02129/) (2023)
- [Adversarial Training for a Continuous Robustness Control Problem in Power Systems](https://mlsanity.com/paper/2012.11390/) (2020)
- [Reinforcement learning for Energies of the future and carbon neutrality: a Challenge Design](https://mlsanity.com/paper/2207.10330/) (2022)

## Built on by (most cited first, 10 of 10)

- [Graph Reinforcement Learning for Power Grids: A Comprehensive Survey](https://mlsanity.com/paper/2407.04522/) (2024)
- [Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control](https://mlsanity.com/paper/2502.08681/) (2025)
- [Imitation Learning for Intra-Day Power Grid Operation through Topology Actions](https://mlsanity.com/paper/2407.19865/) (2024)
- [State and Action Factorization in Power Grids](https://mlsanity.com/paper/2409.04467/) (2024)
- [Fault Detection for agents on power grid topology optimization: A Comprehensive analysis](https://mlsanity.com/paper/2406.16426/) (2024)
- [Learning Topology Actions for Power Grid Control: A Graph-Based Soft-Label Imitation Learning Approach](https://mlsanity.com/paper/2503.15190/) (2025)
- [Power Grid Control with Graph-Based Distributed Reinforcement Learning](https://mlsanity.com/paper/2509.02861/) (2025)
- [Physics Informed Reinforcement Learning with Gibbs Priors for Topology Control in Power Grids](https://mlsanity.com/paper/2604.01830/) (2026)
- [Audited Selective Verification for Risk-Controlled N-1 Thermal Contingency Screening under Deployment Shift](https://mlsanity.com/paper/2607.13221/) (2026)
- [Interpretable Policy Distillation for Power Grid Topology Control](https://mlsanity.com/paper/2606.00561/) (2026)

## Abstract (arXiv)

With the growth of Renewable Energy (RE) generation, the operation of power grids has become increasingly complex. One solution could be automated grid operation, where Deep Reinforcement Learning (DRL) has repeatedly shown significant potential in Learning to Run a Power Network (L2RPN) challenges. However, only individual actions at the substation level have been subjected to topology optimization by most existing DRL algorithms. In contrast, we propose a more holistic approach by proposing specific Target Topologies (TTs) as actions. These topologies are selected based on their robustness. As part of this paper, we present a search algorithm to find the TTs and upgrade our previously developed DRL agent CurriculumAgent (CAgent) to a novel topology agent. We compare the upgrade to the previous CAgent and can increase their L2RPN score significantly by 10%. Further, we achieve a 25% better median survival time with our TTs included. Later analysis shows that almost all TTs are close to the base topology, explaining their robustness