# Digital Twin Assisted Deep Reinforcement Learning for Online Admission Control in Sliced Network

arXiv: https://arxiv.org/abs/2310.09299 · HTML: https://mlsanity.com/paper/2310.09299/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 59 references that are in the map.
- First posted on arXiv 7 Oct 2023.

## Builds on (most cited first, 18 of 18)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [Single and Multi-Agent Deep Reinforcement Learning for AI-Enabled Wireless Networks: A Tutorial](https://mlsanity.com/paper/2011.03615/) (2020)
- [Edge Learning for B5G Networks with Distributed Signal Processing: Semantic Communication, Edge Computing, and Wireless Sensing](https://mlsanity.com/paper/2206.00422/) (2022)
- [Assessing Transferability from Simulation to Reality for Reinforcement Learning](https://mlsanity.com/paper/1907.04685/) (2019)
- [Optimal and Fast Real-time Resources Slicing with Deep Dueling Neural Networks](https://mlsanity.com/paper/1902.09696/) (2019)
- [Bypassing the Simulation-to-reality Gap: Online Reinforcement Learning using a Supervisor](https://mlsanity.com/paper/2209.11082/) (2022)
- [Semantic-aware Digital Twin for Metaverse: A Comprehensive Review](https://mlsanity.com/paper/2305.18304/) (2023)
- [PSACCF: Prioritized Online Slice Admission Control Considering Fairness in 5G/B5G Networks](https://mlsanity.com/paper/2202.05447/) (2022)
- [Learning to Optimize Resource Assignment for Task Offloading in Mobile Edge Computing](https://mlsanity.com/paper/2203.09954/) (2022)
- [Deep Learning-Based Modeling of 5G Core Control Plane for 5G Network Digital Twin](https://mlsanity.com/paper/2302.06980/) (2023)
- [A Hybrid Framework of Reinforcement Learning and Convex Optimization for UAV-Based Autonomous Metaverse Data Collection](https://mlsanity.com/paper/2305.18481/) (2023)
- [Deep Reinforcement Learning for End-to-End Network Slicing: Challenges and Solutions](https://mlsanity.com/paper/2207.14766/) (2022)

## Built on by (most cited first, 5 of 5)

- [Wireless Network Digital Twin for 6G: Generative AI as A Key Enabler](https://mlsanity.com/paper/2311.17451/) (2023)
- [Large Vision Model-Enhanced Digital Twin with Deep Reinforcement Learning for User Association and Load Balancing in Dynamic Wireless Networks](https://mlsanity.com/paper/2410.07611/) (2024)
- [Provable Performance Bounds for Digital Twin-driven Deep Reinforcement Learning in Wireless Networks: A Novel Digital-Twin Bisimulation Metric](https://mlsanity.com/paper/2502.17983/) (2025)
- [Optimizing Reinforcement Learning Training over Digital Twin Enabled Multi-fidelity Networks](https://mlsanity.com/paper/2603.08931/) (2026)
- [Should I Use This Synthetic Dataset for Training? How to Test with Minimal Real Data](https://mlsanity.com/paper/2608.27996/) (2026)

## Abstract (arXiv)

The proliferation of diverse wireless services in 5G and beyond has led to the emergence of network slicing technologies. Among these, admission control plays a crucial role in achieving service-oriented optimization goals through the selective acceptance of service requests. Although deep reinforcement learning (DRL) forms the foundation in many admission control approaches thanks to its effectiveness and flexibility, initial instability with excessive convergence delay of DRL models hinders their deployment in real-world networks. We propose a digital twin (DT) accelerated DRL solution to address this issue. Specifically, we first formulate the admission decision-making process as a semi-Markov decision process, which is subsequently simplified into an equivalent discrete-time Markov decision process to facilitate the implementation of DRL methods. A neural network-based DT is established with a customized output layer for queuing systems, trained through supervised learning, and then employed to assist the training phase of the DRL model. Extensive simulations show that the DT-accelerated DRL improves resource utilization by over 40% compared to the directly trained state-of-the-art dueling deep Q-learning model. This improvement is achieved while preserving the model's capability to optimize the long-term rewards of the admission process.