# Learning When to Automate: Queue Control in Human-AI Service Systems

arXiv: https://arxiv.org/abs/2607.06017 · HTML: https://mlsanity.com/paper/2607.06017/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 37 references that are in the map.
- First posted on arXiv 7 Jul 2026.

## Builds on (most cited first, 10 of 10)

- [Online Convex Optimization with Stochastic Constraints](https://mlsanity.com/paper/1708.03741/) (2017)
- [Learning Unknown Service Rates in Queues: A Multi-Armed Bandit Approach](https://mlsanity.com/paper/1604.06377/) (2016)
- [Learning While Scheduling in Multi-Server Systems with Unknown Statistics: MaxWeight with Discounted UCB](https://mlsanity.com/paper/2209.01126/) (2022)
- [Time to Transfer: Predicting and Evaluating Machine-Human Chatting Handoff](https://mlsanity.com/paper/2012.07610/) (2020)
- [A Lyapunov-Based Methodology for Constrained Optimization with Bandit Feedback](https://mlsanity.com/paper/2106.05165/) (2021)
- [A Role-Selected Sharing Network for Joint Machine-Human Chatting Handoff and Service Satisfaction Analysis](https://mlsanity.com/paper/2109.08412/) (2021)
- [Agentic AI and Human-in-the-Loop Interventions: Field Experimental Evidence from Alibaba's Customer Service Operations](https://mlsanity.com/paper/2605.14830/) (2026)
- [One System to Rule them All: a Universal Intent Recognition System for Customer Service Chatbots](https://mlsanity.com/paper/2112.08261/) (2021)
- [The Transient Cost of Learning in Queueing Systems](https://mlsanity.com/paper/2308.07817/) (2023)
- [Drift Plus Optimistic Penalty: A Learning Framework for Stochastic Network Optimization with Improved Regret Bounds](https://mlsanity.com/paper/2509.03762/) (2025)

## Abstract (arXiv)

We study a human-AI service system in which tasks arrive sequentially and are processed through a two-stage architecture: an automated chatbot followed, when necessary, by a human agent. We consider T sequentially arriving tasks, each belonging to one of K heterogeneous types. For each task the decision maker chooses how many resources to allocate to the chatbot, whose type-dependent success probabilities are initially unknown. Tasks not resolved by the chatbot enter type-dependent human-service queues, where they are processed by a human agent with unknown service rates. This model captures a central tradeoff in hybrid service systems: relying more on automation reduces human congestion but increases chatbot costs, while insufficient automation may overload the human agent. We propose the UCB-DPP policy, which combines Upper Confidence Bounds with Drift-Plus-Penalty control to learn the unknown parameters of the system while making queue-aware decisions. We prove that UCB-DPP achieves regret widetildeO(K√T) and guarantees mean-rate stability of the human-service queues. Simulations on synthetic instances show that the proposed policy outperforms natural baselines.