# Deep Reinforcement Learning with Discrete Normalized Advantage Functions for Resource Management in Network Slicing

arXiv: https://arxiv.org/abs/1906.04594 · HTML: https://mlsanity.com/paper/1906.04594/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2019.

## Built on by (most cited first, 6 of 6)

- [A Survey on Large-Population Systems and Scalable Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2209.03859/) (2022)
- [Self-play Learning Strategies for Resource Assignment in Open-RAN Networks](https://mlsanity.com/paper/2103.02649/) (2021)
- [Decentralized Federated Reinforcement Learning for User-Centric Dynamic TFDD Control](https://mlsanity.com/paper/2211.02296/) (2022)
- [Federated learning and next generation wireless communications: A survey on bidirectional relationship](https://mlsanity.com/paper/2110.07649/) (2021)
- [To Trust or Not to Trust: On Calibration in ML-based Resource Allocation for Wireless Networks](https://mlsanity.com/paper/2507.17494/) (2025)
- [Privacy-Aware Agent Collaboration for Dynamic VR Slice Management in 6G SD-RAN](https://mlsanity.com/paper/2606.26123/) (2026)

## Abstract (arXiv)

Network slicing promises to provision diversified services with distinct requirements in one infrastructure. Deep reinforcement learning (e.g., deep Q-learning, DQL) is assumed to be an appropriate algorithm to solve the demand-aware inter-slice resource management issue in network slicing by regarding the varying demands and the allocated bandwidth as the environment state and the action, respectively. However, allocating bandwidth in a finer resolution usually implies larger action space, and unfortunately DQL fails to quickly converge in this case. In this paper, we introduce discrete normalized advantage functions (DNAF) into DQL, by separating the Q-value function as a state-value function term and an advantage term and exploiting a deterministic policy gradient descent (DPGD) algorithm to avoid the unnecessary calculation of Q-value for every state-action pair. Furthermore, as DPGD only works in continuous action space, we embed a k-nearest neighbor algorithm into DQL to quickly find a valid action in the discrete space nearest to the DPGD output. Finally, we verify the faster convergence of the DNAF-based DQL through extensive simulations.