# LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routing

arXiv: https://arxiv.org/abs/2601.07206 · HTML: https://mlsanity.com/paper/2601.07206/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 62 references that are in the map.
- First posted on arXiv 12 Jan 2026.

## Builds on (most cited first, 30 of 39)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Let's Verify Step by Step](https://mlsanity.com/paper/2305.20050/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools](https://mlsanity.com/paper/2406.12793/) (2024)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [Towards General Text Embeddings with Multi-stage Contrastive Learning](https://mlsanity.com/paper/2308.03281/) (2023)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion](https://mlsanity.com/paper/2306.02561/) (2023)
- [FinQA: A Dataset of Numerical Reasoning over Financial Data](https://mlsanity.com/paper/2109.00122/) (2021)
- [From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline](https://mlsanity.com/paper/2406.11939/) (2024)
- [GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models](https://mlsanity.com/paper/2508.06471/) (2025)
- [Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing](https://mlsanity.com/paper/2404.14618/) (2024)
- [GraphRouter: A Graph-based Router for LLM Selections](https://mlsanity.com/paper/2410.03834/) (2024)
- [On Memorization of Large Language Models in Logical Reasoning](https://mlsanity.com/paper/2410.23123/) (2024)
- [MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmark](https://mlsanity.com/paper/2405.12209/) (2024)
- [Universal Model Routing for Efficient LLM Inference](https://mlsanity.com/paper/2502.08773/) (2025)
- [KOR-Bench: Benchmarking Language Models on Knowledge-Orthogonal Reasoning Tasks](https://mlsanity.com/paper/2410.06526/) (2024)
- [UltraMedical: Building Specialized Generalists in Biomedicine](https://mlsanity.com/paper/2406.03949/) (2024)
- [BEST-Route: Adaptive LLM Routing with Test-Time Optimal Compute](https://mlsanity.com/paper/2506.22716/) (2025)
- [Are Your LLMs Capable of Stable Reasoning?](https://mlsanity.com/paper/2412.13147/) (2024)

## Built on by (most cited first, 30 of 38)

- [Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey](https://mlsanity.com/paper/2603.04445/) (2026)
- [REFLEX with Jev for Efficient Selective Control in LLM Agents](https://mlsanity.com/paper/2609.26532/) (2026)
- [Agent-as-a-Router: Agentic Model Routing for Coding Tasks](https://mlsanity.com/paper/2606.22902/) (2026)
- [Uno-Orchestra: Parsimonious Agent Routing via Selective Delegation](https://mlsanity.com/paper/2605.05007/) (2026)
- [TwinRouterBench: Fast Static and Live Dynamic Evaluation for Realistic Agentic LLM Routing](https://mlsanity.com/paper/2605.18859/) (2026)
- [LLM Router: Rethinking Routing with Prefill Activations](https://mlsanity.com/paper/2603.20895/) (2026)
- [What Does a Routing Oracle Measure Under Stochastic Decoding? Coupling, Scorer Choice, and Single-Commit Ceilings](https://mlsanity.com/paper/2607.03436/) (2026)
- [Agentic Routing: The Harness-Native Data Flywheel](https://mlsanity.com/paper/2607.11399/) (2026)
- [Most of the LLM Routing Gap Is Task Type](https://mlsanity.com/paper/2608.23023/) (2026)
- [Switchcraft: AI Model Router for Agentic Tool Calling](https://mlsanity.com/paper/2605.07112/) (2026)
- [The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Project](https://mlsanity.com/paper/2603.21354/) (2026)
- [Latency-Quality Routing for Functionally Equivalent Tools in LLM Agents](https://mlsanity.com/paper/2605.14241/) (2026)
- [The Routing Plateau: Understanding the Accuracy Limits of LLM Routers](https://mlsanity.com/paper/2606.07587/) (2026)
- [Resample or Reroute? Recoverable Stopping Debt Without Identified Action Selection](https://mlsanity.com/paper/2607.08665/) (2026)
- [VDAR-Router: Adaptive LLMs Routing via Verbalized Query Difficulty Analysis Retrieval](https://mlsanity.com/paper/2607.18098/) (2026)
- [TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI](https://mlsanity.com/paper/2607.22465/) (2026)
- [Signed Rescue Routing: Harm-Aware Cascades for Efficient LLM Inference](https://mlsanity.com/paper/2609.07786/) (2026)
- [Control the Harness, Control the Cost: Routing and Governing AI Coding Agents in the Enterprise](https://mlsanity.com/paper/2609.28919/) (2026)
- [A Large-Scale Benchmark and Risk Assessment of Traffic Analysis Attacks on Cloud LLM Services](https://mlsanity.com/paper/2609.31877/) (2026)
- [RSI-Router: Evolving Subtask-Level LLM Routing and Skills for Cost-Efficient Agents](https://mlsanity.com/paper/2609.34712/) (2026)
- [Dynamic LLM Routers are Often Misguided](https://mlsanity.com/paper/2610.02762/) (2026)
- [Model Routing as a Trust Problem: Route Receipts for Adaptive AI Systems](https://mlsanity.com/paper/2605.01710/) (2026)
- [LLMoxie: Exploring Agentic AI for Scientific Software Development](https://mlsanity.com/paper/2607.02703/) (2026)
- [Aleena: Alignment Agent for Research Software Engineering Collaborations](https://mlsanity.com/paper/2607.08043/) (2026)
- [Detecting an Effect Is Not Learning to Act on It: A Reward-SNR Floor for LLM Acquisition Agents](https://mlsanity.com/paper/2608.10441/) (2026)
- [SCOPE-Router: Cost-Aware Open-Set VLM Routing for Execution-Oriented Tasks](https://mlsanity.com/paper/2608.12127/) (2026)
- [RLCascadeRouter: Quality-Estimator-Free Cascade Routing via Reinforcement Learning](https://mlsanity.com/paper/2608.15817/) (2026)
- [Which LLM for Which Work? Budgeted Model Allocation under Uncertain Evaluation](https://mlsanity.com/paper/2608.29560/) (2026)
- [Mo' Models, Mo' Problems: How to best select model pools when designing Multi-Agent Systems](https://mlsanity.com/paper/2609.17306/) (2026)
- [Measured Joules, Learned Routes: Learning to Route for Energy-Efficient LLM Serving](https://mlsanity.com/paper/2609.23085/) (2026)

## Abstract (arXiv)

Large language model (LLM) routing assigns each query to the most suitable model from an ensemble. We introduce LLMRouterBench, a large-scale benchmark and unified framework for LLM routing. It comprises over 400K instances from 21 datasets and 33 models. Moreover, it provides comprehensive metrics for both performance-oriented routing and performance-cost trade-off routing, and integrates 10 representative routing baselines. Using LLMRouterBench, we systematically re-evaluate the field. While confirming strong model complementarity-the central premise of LLM routing-we find that many routing methods exhibit similar performance under unified evaluation, and several recent approaches, including commercial routers, fail to reliably outperform a simple baseline. Meanwhile, a substantial gap remains to the Oracle, driven primarily by persistent model-recall failures. We further show that backbone embedding models have limited impact, that larger ensembles exhibit diminishing returns compared to careful model curation, and that the benchmark also enables latency-aware analysis. All code and data are available at https://github.com/ynulihao/LLMRouterBench.