# Reinforcement Learning for Dynamic Resource Allocation in Optical Networks: Hype or Hope?

arXiv: https://arxiv.org/abs/2502.12804 · HTML: https://mlsanity.com/paper/2502.12804/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 161 references that are in the map.
- First posted on arXiv 18 Feb 2025.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 12 of 12)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Pointer Networks](https://mlsanity.com/paper/1506.03134/) (2015)
- [Deep Reinforcement Learning that Matters](https://mlsanity.com/paper/1709.06560/) (2017)
- [Implementation Matters in Deep Policy Gradients: A Case Study on PPO and TRPO](https://mlsanity.com/paper/2005.12729/) (2020)
- [A Closer Look at Invalid Action Masking in Policy Gradient Algorithms](https://mlsanity.com/paper/2006.14171/) (2020)
- [Deep Reinforcement Learning meets Graph Neural Networks: exploring a routing optimization use case](https://mlsanity.com/paper/1910.07421/) (2019)
- [DeepRMSA: A Deep Reinforcement Learning Framework for Routing, Modulation and Spectrum Assignment in Elastic Optical Networks](https://mlsanity.com/paper/1905.02248/) (2019)
- [OpticGAI: Generative AI-aided Deep Reinforcement Learning for Optical Networks Optimization](https://mlsanity.com/paper/2406.15906/) (2024)
- [Resource Allocation in Multicore Elastic Optical Networks: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2207.02074/) (2022)
- [Launch Power Optimization for Dynamic Elastic Optical Networks over C+L Bands](https://mlsanity.com/paper/2308.13578/) (2023)
- [DRL-Assisted Dynamic QoT-Aware Service Provisioning in Multi-Band Elastic Optical Networks](https://mlsanity.com/paper/2408.03221/) (2024)

## Abstract (arXiv)

The application of reinforcement learning (RL) to dynamic resource allocation in optical networks has been the focus of intense research activity in recent years, with almost 100 peer-reviewed papers. We present a review of progress in the field, and identify significant gaps in benchmarking practices and reproducibility. To determine the strongest benchmark algorithms, we systematically evaluate several heuristics across diverse network topologies. We find that path count and sort criteria for path selection significantly affect the benchmark performance. We meticulously recreate the problems from five landmark papers and apply the improved benchmarks. Our comparisons demonstrate that simple heuristics consistently match or outperform the published RL solutions, often with an order of magnitude lower blocking probability. Furthermore, we present empirical lower bounds on network blocking using a novel defragmentation-based method, revealing that potential improvements over the benchmark heuristics are limited to 19-36% increased traffic load for the same blocking performance in our examples. We make our simulation framework and results publicly available to promote reproducible research and standardized evaluation https://doi.org/10.5281/zenodo.12594495.