# Lookahead Routing for Large Language Models

arXiv: https://arxiv.org/abs/2510.19506 · HTML: https://mlsanity.com/paper/2510.19506/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Oct 2025.

## Built on by (most cited first, 5 of 5)

- [ParetoBandit: Budget-Paced Adaptive Routing for Non-Stationary LLM Serving](https://mlsanity.com/paper/2604.00136/) (2026)
- [Learning Agent Routing From Early Experience](https://mlsanity.com/paper/2605.07180/) (2026)
- [Cost-Optimal LLM Routing with Limited User Feedback under User Satisfaction Guarantees](https://mlsanity.com/paper/2606.19376/) (2026)
- [Routesplain: Towards Faithful and Intervenable Routing for Software-related Tasks](https://mlsanity.com/paper/2511.09373/) (2025)
- [Resample or Reroute? Recoverable Stopping Debt Without Identified Action Selection](https://mlsanity.com/paper/2607.08665/) (2026)

## Abstract (arXiv)

Large language model (LLM) routers improve the efficiency of multi-model systems by directing each query to the most appropriate model while leveraging the diverse strengths of heterogeneous LLMs. Most existing approaches frame routing as a classification problem based solely on the input query. While this reduces overhead by avoiding inference across all models, it overlooks valuable information that could be gleaned from potential outputs and fails to capture implicit intent or contextual nuances that often emerge only during response generation. These limitations can result in suboptimal routing decisions, particularly for complex or ambiguous queries that require deeper semantic understanding. To address this challenge, we propose Lookahead, a routing framework that "foresees" potential model outputs by predicting their latent representations and uses these predictions to guide model selection, thus enabling more informed routing without full inference. Within this framework, we implement two approaches based on causal and masked language models. Empirical evaluations across seven public benchmarks - spanning instruction following, mathematical reasoning, and code generation - show that Lookahead consistently outperforms existing routing baselines, achieving an average performance gain of 7.7% over the state-of-the-art. Our code is available at https://github.com/huangcb01/lookahead-routing.