# Digital Twin-Assisted Efficient Reinforcement Learning for Edge Task Scheduling

arXiv: https://arxiv.org/abs/2208.01781 · HTML: https://mlsanity.com/paper/2208.01781/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 11 references that are in the map.
- First posted on arXiv 2 Aug 2022.

## Builds on (most cited first, 2 of 2)

- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Deep Reinforcement Learning for Delay-Oriented IoT Task Scheduling in Space-Air-Ground Integrated Network](https://mlsanity.com/paper/2010.01471/) (2020)

## Built on by (most cited first, 11 of 11)

- [A Bayesian Framework for Digital Twin-Based Control, Monitoring, and Data Collection in Wireless Systems](https://mlsanity.com/paper/2212.01351/) (2022)
- [Toward Enhanced Reinforcement Learning-Based Resource Management via Digital Twin: Opportunities, Applications, and Challenges](https://mlsanity.com/paper/2406.07857/) (2024)
- [Scalable Resource Management for Dynamic MEC: An Unsupervised Link-Output Graph Neural Network Approach](https://mlsanity.com/paper/2306.08938/) (2023)
- [Digital Twin-Assisted Knowledge Distillation Framework for Heterogeneous Federated Learning](https://mlsanity.com/paper/2303.06155/) (2023)
- [Digital Twin-Based Multiple Access Optimization and Monitoring via Model-Driven Bayesian Learning](https://mlsanity.com/paper/2210.05582/) (2022)
- [Imperfect Digital Twin Assisted Low Cost Reinforcement Training for Multi-UAV Networks](https://mlsanity.com/paper/2310.16302/) (2023)
- [Knowledge-Driven Multi-Agent Reinforcement Learning for Computation Offloading in Cybertwin-Enabled Internet of Vehicles](https://mlsanity.com/paper/2308.02603/) (2023)
- [Effectively Heterogeneous Federated Learning: A Pairing and Split Learning Based Approach](https://mlsanity.com/paper/2308.13849/) (2023)
- [Constructing and Evaluating Digital Twins: An Intelligent Framework for DT Development](https://mlsanity.com/paper/2406.13145/) (2024)
- [Trapezoidal Gradient Descent for Effective Reinforcement Learning in Spiking Networks](https://mlsanity.com/paper/2406.13568/) (2024)
- [TwinLoop: Simulation-in-the-Loop Digital Twins for Online Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2604.06610/) (2026)

## Abstract (arXiv)

Task scheduling is a critical problem when one user offloads multiple different tasks to the edge server. When a user has multiple tasks to offload and only one task can be transmitted to server at a time, while server processes tasks according to the transmission order, the problem is NP-hard. However, it is difficult for traditional optimization methods to quickly obtain the optimal solution, while approaches based on reinforcement learning face with the challenge of excessively large action space and slow convergence. In this paper, we propose a Digital Twin (DT)-assisted RL-based task scheduling method in order to improve the performance and convergence of the RL. We use DT to simulate the results of different decisions made by the agent, so that one agent can try multiple actions at a time, or, similarly, multiple agents can interact with environment in parallel in DT. In this way, the exploration efficiency of RL can be significantly improved via DT, and thus RL can converges faster and local optimality is less likely to happen. Particularly, two algorithms are designed to made task scheduling decisions, i.e., DT-assisted asynchronous Q-learning (DTAQL) and DT-assisted exploring Q-learning (DTEQL). Simulation results show that both algorithms significantly improve the convergence speed of Q-learning by increasing the exploration efficiency.