# Leveraging Uncertainty Estimation for Efficient LLM Routing

arXiv: https://arxiv.org/abs/2502.11021 · HTML: https://mlsanity.com/paper/2502.11021/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 16 Feb 2025.

## Builds on (most cited first, 15 of 15)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [Latent Retrieval for Weakly Supervised Open Domain Question Answering](https://mlsanity.com/paper/1906.00300/) (2019)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [GraphRouter: A Graph-based Router for LLM Selections](https://mlsanity.com/paper/2410.03834/) (2024)
- [TensorOpera Router: A Multi-Model Router for Efficient LLM Inference](https://mlsanity.com/paper/2408.12320/) (2024)
- [MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMs](https://mlsanity.com/paper/2402.11756/) (2024)
- [Do Not Design, Learn: A Trainable Scoring Function for Uncertainty Estimation in Generative LLMs](https://mlsanity.com/paper/2406.11278/) (2024)

## Built on by (most cited first, 21 of 21)

- [Harnessing Multiple Large Language Models: A Survey on LLM Ensemble](https://mlsanity.com/paper/2502.18036/) (2025)
- [Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey](https://mlsanity.com/paper/2603.04445/) (2026)
- [When Routing Collapses: On the Degenerate Convergence of LLM Routers](https://mlsanity.com/paper/2602.03478/) (2026)
- [Walking Through Uncertainty: An Empirical Study of Uncertainty Estimation for Audio-Aware Large Language Models](https://mlsanity.com/paper/2604.25591/) (2026)
- [Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators](https://mlsanity.com/paper/2509.03647/) (2025)
- [TCAndon-Router: Adaptive Reasoning Router for Multi-Agent Collaboration](https://mlsanity.com/paper/2601.04544/) (2026)
- [Towards Fair and Comprehensive Evaluation of Routers in Collaborative LLM Systems](https://mlsanity.com/paper/2602.11877/) (2026)
- [Early-Exit and Instant Confidence Translation Quality Estimation](https://mlsanity.com/paper/2502.14429/) (2025)
- [SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models](https://mlsanity.com/paper/2505.07680/) (2025)
- [Balancing Information Accuracy and Response Timeliness in Networked LLMs](https://mlsanity.com/paper/2508.02209/) (2025)
- [LatentRouter: Can We Choose the Right Multimodal Model Before Seeing Its Answer?](https://mlsanity.com/paper/2605.11301/) (2026)
- [A Systematic Evaluation of Black-Box Uncertainty Estimation Methods for Large Language Models](https://mlsanity.com/paper/2606.19868/) (2026)
- [Meta-Router: Bridging Gold-standard and Preference-based Evaluations in Large Language Model Routing](https://mlsanity.com/paper/2509.25535/) (2025)
- [Zero-Shot Confidence Estimation for Small LLMs: When Supervised Baselines Aren't Worth Training](https://mlsanity.com/paper/2605.02241/) (2026)
- [FlyRoute: Self-Evolving Agent Profiling via Data Flywheel for Adaptive Task Routing](https://mlsanity.com/paper/2605.22057/) (2026)
- [CoE-Ops: Collaboration of LLM-based Experts for AIOps Question-Answering](https://mlsanity.com/paper/2507.22937/) (2025)
- [Conditional Performance Guarantee for Large Reasoning Models](https://mlsanity.com/paper/2601.22790/) (2026)
- [LaCy: What Small Language Models Can and Should Learn is Not Just a Question of Loss](https://mlsanity.com/paper/2602.12005/) (2026)
- [AgentOpt v0.1 Technical Report: Client-Side Optimization for LLM-Based Agent](https://mlsanity.com/paper/2604.06296/) (2026)
- [FrankenReport: Early Exiting in Long-Form Generation Using Expected Value of Computation](https://mlsanity.com/paper/2609.06320/) (2026)
- [Auditing and Repairing LLM-as-Judge Failures in a Production Text-to-SQL Pipeline](https://mlsanity.com/paper/2609.30290/) (2026)

## Abstract (arXiv)

Deploying large language models (LLMs) in edge-cloud environments requires an efficient routing strategy to balance cost and response quality. Traditional approaches prioritize either human-preference data or accuracy metrics from benchmark datasets as routing criteria, but these methods suffer from rigidity and subjectivity. Moreover, existing routing frameworks primarily focus on accuracy and cost, neglecting response quality from a human preference perspective. In this work, we propose the Confidence-Driven LLM Router, a novel framework that leverages uncertainty estimation to optimize routing decisions. To comprehensively assess routing performance, we evaluate both system cost efficiency and response quality. In particular, we introduce the novel use of LLM-as-a-Judge to simulate human rating preferences, providing the first systematic assessment of response quality across different routing strategies. Extensive experiments on MT-Bench, GSM8K, and MMLU demonstrate that our approach outperforms state-of-the-art routing methods, achieving superior response quality while maintaining cost efficiency.