# Online Learning and Optimization for Queues with Unknown Demand Curve and Service Distribution

arXiv: https://arxiv.org/abs/2303.03399 · HTML: https://mlsanity.com/paper/2303.03399/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 33 references that are in the map.
- First posted on arXiv 6 Mar 2023.

## Builds on (most cited first, 5 of 5)

- [Queueing Network Controls via Deep Reinforcement Learning](https://mlsanity.com/paper/2008.01644/) (2020)
- [Learning Unknown Service Rates in Queues: A Multi-Armed Bandit Approach](https://mlsanity.com/paper/1604.06377/) (2016)
- [Stable Reinforcement Learning with Unbounded State Space](https://mlsanity.com/paper/2006.04353/) (2020)
- [Learning and Information in Stochastic Networks and Queues](https://mlsanity.com/paper/2105.08769/) (2021)
- [An online learning approach to dynamic pricing and capacity sizing in service systems](https://mlsanity.com/paper/2009.02911/) (2020)

## Built on by (most cited first, 3 of 3)

- [A Correlation-induced Finite Difference Estimator](https://mlsanity.com/paper/2405.05638/) (2024)
- [Steady-State Analysis and Online Learning for Queues with Hawkes Arrivals](https://mlsanity.com/paper/2311.02577/) (2023)
- [Near-Optimal Regret-Queue Length Tradeoff in Online Learning for Two-Sided Markets](https://mlsanity.com/paper/2510.14097/) (2025)

## Abstract (arXiv)

We investigate an optimization problem in a queueing system where the service provider selects the optimal service fee p and service capacity μ to maximize the cumulative expected profit (the service revenue minus the capacity cost and delay penalty). The conventional predict-then-optimize (PTO) approach takes two steps: first, it estimates the model parameters (e.g., arrival rate and service-time distribution) from data; second, it optimizes a model based on the estimated parameters. A major drawback of PTO is that its solution accuracy can often be highly sensitive to the parameter estimation errors because PTO is unable to properly link these errors (step 1) to the quality of the optimized solutions (step 2). To remedy this issue, we develop an online learning framework that automatically incorporates the aforementioned parameter estimation errors in the solution prescription process; it is an integrated method that can "learn" the optimal solution without needing to set up the parameter estimation as a separate step as in PTO. Effectiveness of our online learning approach is substantiated by (i) theoretical results including the algorithm convergence and analysis of the regret ("cost" to pay over time for the algorithm to learn the optimal policy), and (ii) engineering confirmation via simulation experiments of a variety of representative examples. We also provide careful comparisons for PTO and the online learning method.