# An Overview for Markov Decision Processes in Queues and Networks

arXiv: https://arxiv.org/abs/1907.10243 · HTML: https://mlsanity.com/paper/1907.10243/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2019.

## Built on by (most cited first, 4 of 4)

- [Learning Mean-Field Control for Delayed Information Load Balancing in Large Queuing Systems](https://mlsanity.com/paper/2208.04777/) (2022)
- [Reinforcement Learning in a Birth and Death Process: Breaking the Dependence on the State Space](https://mlsanity.com/paper/2302.10667/) (2023)
- [Performance Evaluation, Optimization and Dynamic Decision in Blockchain Systems: A Recent Overview](https://mlsanity.com/paper/2211.15907/) (2022)
- [RA-CAD: Learning Post-Execution Critique for State-Aware Text-to-CAD Generation](https://mlsanity.com/paper/2608.05714/) (2026)

## Abstract (arXiv)

Markov decision processes (MDPs) in queues and networks have been an interesting topic in many practical areas since the 1960s. This paper provides a detailed overview on this topic and tracks the evolution of many basic results. Also, this paper summarizes several interesting directions in the future research. We hope that this overview can shed light to MDPs in queues and networks, and also to their extensive applications in various practical areas.