# Queue-Learning: A Reinforcement Learning Approach for Providing Quality of Service

arXiv: https://arxiv.org/abs/2101.04627 · HTML: https://mlsanity.com/paper/2101.04627/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 16 references that are in the map.
- First posted on arXiv 12 Jan 2021.

## Builds on (most cited first, 4 of 4)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [Reinforcement Learning-based Admission Control in Delay-sensitive Service Systems](https://mlsanity.com/paper/2008.09590/) (2020)
- [Dynamic Service Rate Control for a Single Server Queue with Markov Modulated Arrivals](https://mlsanity.com/paper/1307.2601/) (2013)
- [Dynamic Scaling of Virtualized, Distributed Service Chains: A Case Study of IMS](https://mlsanity.com/paper/1702.02853/) (2017)

## Built on by (most cited first, 7 of 7)

- [Simple Agent, Complex Environment: Efficient Reinforcement Learning with Agent States](https://mlsanity.com/paper/2102.05261/) (2021)
- [Dynamic Resource Management for Providing QoS in Drone Delivery Systems](https://mlsanity.com/paper/2103.04015/) (2021)
- [Optimal Control of Multiclass Fluid Queueing Networks: A Machine Learning Approach](https://mlsanity.com/paper/2307.12405/) (2023)
- [Intervention-Assisted Policy Gradient Methods for Online Stochastic Queuing Network Optimization: Technical Report](https://mlsanity.com/paper/2404.04106/) (2024)
- [A Novel Switch-Type Policy Network for Resource Allocation Problems: Technical Report](https://mlsanity.com/paper/2501.11136/) (2025)
- [Percentile-Based Deep Reinforcement Learning and Reward Based Personalization For Delay Aware RAN Slicing in O-RAN](https://mlsanity.com/paper/2507.18111/) (2025)
- [Near-Optimal Regret-Queue Length Tradeoff in Online Learning for Two-Sided Markets](https://mlsanity.com/paper/2510.14097/) (2025)

## Abstract (arXiv)

End-to-end delay is a critical attribute of quality of service (QoS) in application domains such as cloud computing and computer networks. This metric is particularly important in tandem service systems, where the end-to-end service is provided through a chain of services. Service-rate control is a common mechanism for providing QoS guarantees in service systems. In this paper, we introduce a reinforcement learning-based (RL-based) service-rate controller that provides probabilistic upper-bounds on the end-to-end delay of the system, while preventing the overuse of service resources. In order to have a general framework, we use queueing theory to model the service systems. However, we adopt an RL-based approach to avoid the limitations of queueing-theoretic methods. In particular, we use Deep Deterministic Policy Gradient (DDPG) to learn the service rates (action) as a function of the queue lengths (state) in tandem service systems. In contrast to existing RL-based methods that quantify their performance by the achieved overall reward, which could be hard to interpret or even misleading, our proposed controller provides explicit probabilistic guarantees on the end-to-end delay of the system. The evaluations are presented for a tandem queueing system with non-exponential inter-arrival and service times, the results of which validate our controller's capability in meeting QoS constraints.