# Digital Twin-Enhanced Deep Reinforcement Learning for Resource Management in Networks Slicing

arXiv: https://arxiv.org/abs/2311.16876 · HTML: https://mlsanity.com/paper/2311.16876/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 42 references that are in the map.
- First posted on arXiv 28 Nov 2023.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Deep Reinforcement Learning for Resource Management in Network Slicing](https://mlsanity.com/paper/1805.06591/) (2018)
- [On the Convergence and Sample Complexity Analysis of Deep Q-Networks with ε-Greedy Exploration](https://mlsanity.com/paper/2310.16173/) (2023)
- [Bootstrap Generalization Ability from Loss Landscape Perspective](https://mlsanity.com/paper/2209.08473/) (2022)
- [Millimeter Wave Communications on Overhead Messenger Wire: Deep Reinforcement Learning-Based Predictive Beam Tracking](https://mlsanity.com/paper/2012.00982/) (2020)

## Built on by (most cited first, 7 of 7)

- [Wireless Network Digital Twin for 6G: Generative AI as A Key Enabler](https://mlsanity.com/paper/2311.17451/) (2023)
- [RMTransformer: Accurate Radio Map Construction and Coverage Prediction](https://mlsanity.com/paper/2501.05190/) (2025)
- [RadioFormer: A Multiple-Granularity Radio Map Estimation Transformer with 1textpertenthousand Spatial Sampling](https://mlsanity.com/paper/2504.19161/) (2025)
- [Provable Performance Bounds for Digital Twin-driven Deep Reinforcement Learning in Wireless Networks: A Novel Digital-Twin Bisimulation Metric](https://mlsanity.com/paper/2502.17983/) (2025)
- [Efficient Beam Selection for ISAC in Cell-Free Massive MIMO via Digital Twin-Assisted Deep Reinforcement Learning](https://mlsanity.com/paper/2506.18560/) (2025)
- [Digital Twin Channel-Enabled Online Resource Allocation for 6G: Principle, Architecture and Application](https://mlsanity.com/paper/2507.19974/) (2025)
- [Dual-Loop Control in DCVerse: Advancing Reliable Deployment of AI in Data Centers via Digital Twins](https://mlsanity.com/paper/2604.07559/) (2026)

## Abstract (arXiv)

Network slicing-based communication systems can dynamically and efficiently allocate resources for diversified services. However, due to the limitation of the network interface on channel access and the complexity of the resource allocation, it is challenging to achieve an acceptable solution in the practical system without precise prior knowledge of the dynamics probability model of the service requests. Existing work attempts to solve this problem using deep reinforcement learning (DRL), however, such methods usually require a lot of interaction with the real environment in order to achieve good results. In this paper, a framework consisting of a digital twin and reinforcement learning agents is present to handle the issue. Specifically, we propose to use the historical data and the neural networks to build a digital twin model to simulate the state variation law of the real environment. Then, we use the data generated by the network slicing environment to calibrate the digital twin so that it is in sync with the real environment. Finally, DRL for slice optimization optimizes its own performance in this virtual pre-verification environment. We conducted an exhaustive verification of the proposed digital twin framework to confirm its scalability. Specifically, we propose to use loss landscapes to visualize the generalization of DRL solutions. We explore a distillation-based optimization scheme for lightweight slicing strategies. In addition, we also extend the framework to offline reinforcement learning, where solutions can be used to obtain intelligent decisions based solely on historical data. Numerical simulation experiments show that the proposed digital twin can significantly improve the performance of the slice optimization strategy.