mlsanityData to 7 Oct 2026

RouteLLM: Learning to Route LLMs with Preference Data

arXiv 2406.18665 ↗First posted 26 Jun 2024cs.LG cs.AI cs.CLcited by 633

Not on the map. Why?

Cited by 166 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) exhibit impressive capabilities across a wide range of tasks, yet the choice of which model to use often involves a trade-off between performance and cost. More powerful models, though effective, come with higher expenses, while less capable models are more cost-effective. To address this dilemma, we propose several efficient router models that dynamically select between a stronger and a weaker LLM during inference, aiming to optimize the balance between cost and response quality. We develop a training framework for these routers leveraging human preference data and data augmentation techniques to enhance performance. Our evaluation on widely-recognized benchmarks shows that our approach significantly reduces costs-by over 2 times in certain cases-without compromising the quality of responses. Interestingly, our router models also demonstrate significant transfer learning capabilities, maintaining their performance even when the strong and weak models are changed at test time. This highlights the potential of these routers to provide a cost-effective yet high-performance solution for deploying LLMs.

Key facts. Cited by 633 papers in the map (data through 7 Oct 2026). Builds on 20 of its 34 references that are in the map. First posted on arXiv 26 Jun 2024.

Builds on

20 of 34 references.

References · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  10. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  12. 2024Mixtral of Expertscited by 1,785
8 more
  1. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  2. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  3. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 719
  4. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  5. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
  6. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
  7. 2023Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Modelscited by 141
  8. 2020FrugalML: How to Use ML Prediction APIs More Accurately and Cheaplycited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 27; 2025: 162; 2026: 444.

633 papers.

Sort:

By area:

  1. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  2. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  3. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  4. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  5. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  6. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  7. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 131
  8. 2025Thinkless: LLM Learns When to Thinkcited by 107
  9. 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMscited by 94
  10. 2025Efficient Reasoning Models: A Surveycited by 93
  11. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  12. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
38 more
  1. 2025Universal Model Routing for Efficient LLM Inferencecited by 80
  2. 2025MasRouter: Learning to Route LLMs for Multi-Agent Systemscited by 80
  3. 2024A Unified Approach to Routing and Cascading for LLMscited by 71
  4. 2024Learning to Route LLMs with Confidence Tokenscited by 66
  5. 2025Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learningcited by 64
  6. 2025BEST-Route: Adaptive LLM Routing with Test-Time Optimal Computecited by 63
  7. 2024TensorOpera Router: A Multi-Model Router for Efficient LLM Inferencecited by 59
  8. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 58
  9. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 58
  10. 2025MixLLM: Dynamic Routing in Mixed Large Language Modelscited by 56
  11. 2025Cache-to-Cache: Direct Semantic Communication Between Large Language Modelscited by 54
  12. 2024A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learningcited by 52
  13. 2025SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoningcited by 47
  14. 2024LLM Inference Serving: Survey of Recent Advances and Opportunitiescited by 46
  15. 2024EmbedLLM: Learning Compact Representations of Large Language Modelscited by 46
  16. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  17. 2025IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theorycited by 40
  18. 2024MetaLLM: A High-performant and Cost-efficient Dynamic Framework for Wrapping LLMscited by 39
  19. 2025Answer Convergence as a Signal for Early Stopping in Reasoningcited by 38
  20. 2025Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Modelscited by 37
  21. 2025The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giantscited by 37
  22. 2025OmniRouter: Budget and Performance Controllable Multi-LLM Routingcited by 35
  23. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  24. 2026LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routingcited by 35
  25. 2024Eagle: Efficient Training-Free Router for Multi-LLM Inferencecited by 34
  26. 2025Internet of Agents: Fundamentals, Applications, and Challengescited by 31
  27. 2024Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmapcited by 30
  28. 2025CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routingcited by 29
  29. 2025AgentRouter: A Knowledge-Graph-Guided LLM Router for Collaborative Multi-Agent Question Answeringcited by 29
  30. 2025Prompt-to-Leaderboardcited by 27
  31. 2025Causal LLM Routing: End-to-End Regret Minimization from Observational Datacited by 27
  32. 2025CoMAS: Co-Evolving Multi-Agent Systems via Interaction Rewardscited by 27
  33. 2025When One LLM Drools, Multi-LLM Collaboration Rulescited by 25
  34. 2025Shorten After You're Right: Lazy Length Penalties for Reasoning RLcited by 25
  35. 2025Route to Reason: Adaptive Routing for LLM and Reasoning Strategy Selectioncited by 23
  36. 2026SkillOrchestra: Learning to Route Agents via Skill Transfercited by 22
  37. 2025Leveraging Uncertainty Estimation for Efficient LLM Routingcited by 21
  38. 2025Adaptive LLM Routing under Budget Constraintscited by 21

50 most cited of 633 All 633 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs
  2. 2025LLM Bandit: Cost-Efficient LLM Generation via Preference-Conditioned Dynamic Routing
  3. 2025Multilingual Routing in Mixture-of-Experts
  4. 2025Lookahead Routing for Large Language Models
  5. 2025Dynamic LLM Routing and Selection based on User Preferences: Balancing Performance, Cost, and Ethics
  6. 2025HierRouter: Coordinated Routing of Specialized Large Language Models via Reinforcement Learning
  7. 2024Optimizing LLMs with Direct Preferences: A Data Efficiency Perspective
  8. 2025Confidence-Guided Stepwise Model Routing for Cost-Efficient Reasoning