# RL-QN: A Reinforcement Learning Framework for Optimal Control of Queueing Systems

arXiv: https://arxiv.org/abs/2011.07401 · HTML: https://mlsanity.com/paper/2011.07401/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2020.

## Built on by (most cited first, 11 of 11)

- [Efficient Reinforcement Learning for Routing Jobs in Heterogeneous Queueing Systems](https://mlsanity.com/paper/2402.01147/) (2024)
- [QGym: Scalable Simulation and Benchmarking of Queuing Network Controllers](https://mlsanity.com/paper/2410.06170/) (2024)
- [Adversarial Network Optimization under Bandit Feedback: Maximizing Utility in Non-Stationary Multi-Hop Networks](https://mlsanity.com/paper/2408.16215/) (2024)
- [Learning to Stabilize Online Reinforcement Learning in Unbounded State Spaces](https://mlsanity.com/paper/2306.01896/) (2023)
- [Semi-Gradient SARSA Routing with Theoretical Guarantee on Traffic Stability and Weight Convergence](https://mlsanity.com/paper/2503.14927/) (2025)
- [Natural Policy Gradient for Average Reward Non-Stationary RL](https://mlsanity.com/paper/2504.16415/) (2025)
- [Optimising Call Centre Operations using Reinforcement Learning: Value Iteration versus Proximal Policy Optimisation](https://mlsanity.com/paper/2507.18398/) (2025)
- [A Flexible Multi-Agent Deep Reinforcement Learning Framework for Dynamic Routing and Scheduling of Latency-Critical Services](https://mlsanity.com/paper/2510.11535/) (2025)
- [Near-Optimal Regret-Queue Length Tradeoff in Online Learning for Two-Sided Markets](https://mlsanity.com/paper/2510.14097/) (2025)
- [A Constrained RL Approach for Cost-Efficient Delivery of Latency-Sensitive Applications](https://mlsanity.com/paper/2603.04353/) (2026)
- [Multi-Robot Multi-Queue Control via Exhaustive Assignment Actor-Critic Learning](https://mlsanity.com/paper/2604.03605/) (2026)

## Abstract (arXiv)

With the rapid advance of information technology, network systems have become increasingly complex and hence the underlying system dynamics are often unknown or difficult to characterize. Finding a good network control policy is of significant importance to achieve desirable network performance (e.g., high throughput or low delay). In this work, we consider using model-based reinforcement learning (RL) to learn the optimal control policy for queueing networks so that the average job delay (or equivalently the average queue backlog) is minimized. Traditional approaches in RL, however, cannot handle the unbounded state spaces of the network control problem. To overcome this difficulty, we propose a new algorithm, called Reinforcement Learning for Queueing Networks (RL-QN), which applies model-based RL methods over a finite subset of the state space, while applying a known stabilizing policy for the rest of the states. We establish that the average queue backlog under RL-QN with an appropriately constructed subset can be arbitrarily close to the optimal result. We evaluate RL-QN in dynamic server allocation, routing and switching problems. Simulation results show that RL-QN minimizes the average queue backlog effectively.