# Effective Multi-User Delay-Constrained Scheduling with Deep Recurrent Reinforcement Learning

arXiv: https://arxiv.org/abs/2208.14074 · HTML: https://mlsanity.com/paper/2208.14074/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 36 references that are in the map.
- First posted on arXiv 30 Aug 2022.

## Builds on (most cited first, 8 of 8)

- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Memory-based control with recurrent neural networks](https://mlsanity.com/paper/1512.04455/) (2015)
- [Multi-Agent Deep Reinforcement Learning for Dynamic Power Allocation in Wireless Networks](https://mlsanity.com/paper/1808.00490/) (2018)
- [Softmax Deep Double Deterministic Policy Gradients](https://mlsanity.com/paper/2010.09177/) (2020)
- [Throughput Optimal Decentralized Scheduling of Multi-Hop Networks with End-to-End Deadline Constraints: Unreliable Links](https://mlsanity.com/paper/1606.01608/) (2016)
- [When Backpressure Meets Predictive Scheduling](https://mlsanity.com/paper/1309.1110/) (2013)
- [Timely-Throughput Optimal Scheduling with Prediction](https://mlsanity.com/paper/1712.05677/) (2017)

## Built on by (most cited first, 3 of 3)

- [Asynchronous Federated Reinforcement Learning with Policy Gradient Updates: Algorithm Design and Convergence Analysis](https://mlsanity.com/paper/2404.08003/) (2024)
- [A Deep Recurrent-Reinforcement Learning Method for Intelligent AutoScaling of Serverless Functions](https://mlsanity.com/paper/2308.05937/) (2023)
- [A Flexible Multi-Agent Deep Reinforcement Learning Framework for Dynamic Routing and Scheduling of Latency-Critical Services](https://mlsanity.com/paper/2510.11535/) (2025)

## Abstract (arXiv)

Multi-user delay constrained scheduling is important in many real-world applications including wireless communication, live streaming, and cloud computing. Yet, it poses a critical challenge since the scheduler needs to make real-time decisions to guarantee the delay and resource constraints simultaneously without prior information of system dynamics, which can be time-varying and hard to estimate. Moreover, many practical scenarios suffer from partial observability issues, e.g., due to sensing noise or hidden correlation. To tackle these challenges, we propose a deep reinforcement learning (DRL) algorithm, named Recurrent Softmax Delayed Deep Double Deterministic Policy Gradient (mathttRSD4), which is a data-driven method based on a Partially Observed Markov Decision Process (POMDP) formulation. mathttRSD4 guarantees resource and delay constraints by Lagrangian dual and delay-sensitive queues, respectively. It also efficiently tackles partial observability with a memory mechanism enabled by the recurrent neural network (RNN) and introduces user-level decomposition and node-level merging to ensure scalability. Extensive experiments on simulated/real-world datasets demonstrate that mathttRSD4 is robust to system dynamics and partially observable environments, and achieves superior performances over existing DRL and non-DRL-based methods.