# Deep Reinforcement Learning for Stochastic Computation Offloading in Digital Twin Networks

arXiv: https://arxiv.org/abs/2011.08430 · HTML: https://mlsanity.com/paper/2011.08430/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2020.

## Built on by (most cited first, 15 of 15)

- [A Comprehensive Review of Digital Twin -- Part 1: Modeling and Twinning Enabling Technologies](https://mlsanity.com/paper/2208.14197/) (2022)
- [A Bayesian Framework for Digital Twin-Based Control, Monitoring, and Data Collection in Wireless Systems](https://mlsanity.com/paper/2212.01351/) (2022)
- [Holistic Network Virtualization and Pervasive Network Intelligence for 6G](https://mlsanity.com/paper/2301.00519/) (2023)
- [Causal Semantic Communication for Digital Twins: A Generalizable Imitation Learning Approach](https://mlsanity.com/paper/2304.12502/) (2023)
- [Multiple Access in the Era of Distributed Computing and Edge Intelligence](https://mlsanity.com/paper/2403.07903/) (2024)
- [Digital Twin in Industries: A Comprehensive Survey](https://mlsanity.com/paper/2412.00209/) (2024)
- [Digital Twin Virtualization with Machine Learning for IoT and Beyond 5G Networks: Research Directions for Security and Optimal Control](https://mlsanity.com/paper/2204.01950/) (2022)
- [Two-Timescale Synchronization and Migration for Digital Twin Networks: A Multi-Agent Deep Reinforcement Learning Approach](https://mlsanity.com/paper/2409.01092/) (2024)
- [Reinforcement Learning-Empowered Mobile Edge Computing for 6G Edge Intelligence](https://mlsanity.com/paper/2201.11410/) (2022)
- [A Lyapunov Drift-Plus-Penalty Method Tailored for Reinforcement Learning with Queue Stability](https://mlsanity.com/paper/2506.04291/) (2025)
- [Digital Twin-Based Multiple Access Optimization and Monitoring via Model-Driven Bayesian Learning](https://mlsanity.com/paper/2210.05582/) (2022)
- [When Digital Twin Meets 6G: Concepts, Obstacles, and Research Prospects](https://mlsanity.com/paper/2409.02008/) (2024)
- [Energy Optimization of Multi-task DNN Inference in MEC-assisted XR Devices: A Lyapunov-Guided Reinforcement Learning Approach](https://mlsanity.com/paper/2501.02572/) (2025)
- [Digital Twin-assisted Reinforcement Learning for Resource-aware Microservice Offloading in Edge Computing](https://mlsanity.com/paper/2403.08687/) (2024)
- [Digital Twin Channel-Enabled Online Resource Allocation for 6G: Principle, Architecture and Application](https://mlsanity.com/paper/2507.19974/) (2025)

## Abstract (arXiv)

The rapid development of Industrial Internet of Things (IIoT) requires industrial production towards digitalization to improve network efficiency. Digital Twin is a promising technology to empower the digital transformation of IIoT by creating virtual models of physical objects. However, the provision of network efficiency in IIoT is very challenging due to resource-constrained devices, stochastic tasks, and resources heterogeneity. Distributed resources in IIoT networks can be efficiently exploited through computation offloading to reduce energy consumption while enhancing data processing efficiency. In this paper, we first propose a new paradigm Digital Twin Networks (DTN) to build network topology and the stochastic task arrival model in IIoT systems. Then, we formulate the stochastic computation offloading and resource allocation problem to minimize the long-term energy efficiency. As the formulated problem is a stochastic programming problem, we leverage Lyapunov optimization technique to transform the original problem into a deterministic per-time slot problem. Finally, we present Asynchronous Actor-Critic (AAC) algorithm to find the optimal stochastic computation offloading policy. Illustrative results demonstrate that our proposed scheme is able to significantly outperforms the benchmarks.