# Exploiting the Alternatives: Coordinated Learning via Hierarchical RL for Dynamic VNEAP

arXiv: https://arxiv.org/abs/2512.05207 · HTML: https://mlsanity.com/paper/2512.05207/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 22 references that are in the map.
- First posted on arXiv 4 Dec 2025.

## Builds on (most cited first, 6 of 6)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [On Orchestrating Virtual Network Functions in NFV](https://mlsanity.com/paper/1503.06377/) (2015)
- [Joint Admission Control and Resource Allocation of Virtual Network Embedding via Hierarchical Deep Reinforcement Learning](https://mlsanity.com/paper/2406.17334/) (2024)
- [FlagVNE: A Flexible and Generalizable Reinforcement Learning Framework for Network Resource Allocation](https://mlsanity.com/paper/2404.12633/) (2024)
- [The Power of Alternatives in Network Embedding](https://mlsanity.com/paper/2505.09753/) (2025)

## Abstract (arXiv)

Virtual Network Embedding (VNE) is a key enabler of network slicing, yet most formulations assume that each Virtual Network Request (VNR) has a fixed topology. Recently, VNE with Alternatives (VNEAP) was introduced to capture malleable VNRs, where each request can be instantiated using one of several functionally equivalent topologies that trade resources differently. This flexibility can improve embedding feasibility, but only if the orchestrator can jointly select suitable alternatives and embed them under dynamic arrivals. This paper proposes HRL-VNEAP, a hierarchical reinforcement learning approach for dynamic VNEAP. A high-level policy selects the most suitable alternative topology (or rejects the request), and a low-level policy embeds the chosen topology onto the substrate network. Experiments on realistic substrate topologies under varying arrival rates show that naive exploitation strategies provide only modest gains, whereas HRL-VNEAP outperforms state of the art approaches, improving acceptance ratio by up to 22%, and net profit by up to 20%. An offline MILP upper bound is also used on tractable instances to quantify the remaining optimality gap.