# Load is not what you should balance: Introducing Prequal

arXiv: https://arxiv.org/abs/2312.10172 · HTML: https://mlsanity.com/paper/2312.10172/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Dec 2023.

## Built on by (most cited first, 2 of 2)

- [Lodestar: An Online-Learning LLM Inference Router](https://mlsanity.com/paper/2606.00946/) (2026)
- [Multi-Turn Distributed Inference with Mixture of Experts for 6G Edge--Cloud Networks](https://mlsanity.com/paper/2607.02522/) (2026)

## Abstract (arXiv)

We present Prequal (Probing to Reduce Queuing and Latency), a load balancer for distributed multi-tenant systems. Prequal aims to minimize real-time request latency in the presence of heterogeneous server capacities and non-uniform, time-varying antagonist load. It actively probes server load to leverage the power-of-d-choices paradigm, extending it with asynchronous and reusable probes. Cutting against received wisdom, Prequal does not balance CPU load, but instead selects servers according to estimated latency and active requests-in-flight (RIF). We explore its major design features on a testbed system and evaluate it on YouTube, where it has been deployed for more than two years. Prequal has dramatically decreased tail latency, error rates, and resource use, enabling YouTube and other production systems at Google to run at much higher utilization.