mlsanityData to 7 Oct 2026

Harnessing the Power of Multiple Minds: Lessons Learned from LLM Routing

arXiv 2405.00467 ↗First posted 1 May 2024cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

With the rapid development of LLMs, it is natural to ask how to harness their capabilities efficiently. In this paper, we explore whether it is feasible to direct each input query to a single most suitable LLM. To this end, we propose LLM routing for challenging reasoning tasks. Our extensive experiments suggest that such routing shows promise but is not feasible in all scenarios, so more robust approaches should be investigated to fill this gap.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 1 May 2024.

Builds on

27 of 38 references.

References · 27

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  7. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  8. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  10. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  11. 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
  12. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
15 more
  1. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  2. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
  3. 2013Learning Factored Representations in a Deep Mixture of Expertscited by 275
  4. 2023Sentiment Analysis in the Era of Large Language Models: A Reality Checkcited by 262
  5. 2023LLM in a flash: Efficient Large Language Model Inference with Limited Memorycited by 170
  6. 2023Large Language Model Routing with Benchmark Datasetscited by 166
  7. 2018Model Evaluation, Model Selection, and Algorithm Selection in Machine Learningcited by 157
  8. 2021SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarizationcited by 131
  9. 2017Routing Networks: Adaptive Selection of Non-linear Functions for Multi-Task Learningcited by 125
  10. 2022Evaluating the Factual Consistency of Large Language Models Through News Summarizationcited by 90
  11. 2023Fast Inference of Mixture-of-Experts Language Models with Offloadingcited by 88
  12. 2022SummaReranker: A Multi-Task Mixture-of-Experts Re-ranking Framework for Abstractive Summarizationcited by 61
  13. 2023Predicting Question-Answering Performance of Large Language Models through Semantic Consistencycited by 28
  14. 2023Chain of Thought Prompting Elicits Knowledge Augmentationcited by 21
  15. 2024What Makes Math Word Problems Challenging for LLMs?cited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 24; 2026: 14.

43 papers.

Sort:

By area:

  1. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  2. 2024RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Modelscited by 84
  3. 2025Universal Model Routing for Efficient LLM Inferencecited by 80
  4. 2025MasRouter: Learning to Route LLMs for Multi-Agent Systemscited by 80
  5. 2025BEST-Route: Adaptive LLM Routing with Test-Time Optimal Computecited by 63
  6. 2024TensorOpera Router: A Multi-Model Router for Efficient LLM Inferencecited by 59
  7. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 58
  8. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 43
  9. 2025CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routingcited by 29
  10. 2025Capability Instruction Tuning: A New Paradigm for Dynamic LLM Routingcited by 25
  11. 2025Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in Large Language Model-Based Systemscited by 18
  12. 2025Learning to Deliberate: Meta-policy Collaboration for Agentic LLMs with Multi-agent Reinforcement Learningcited by 18
31 more
  1. 2026EvoRoute: Experience-Driven Self-Routing LLM Agent Systemscited by 13
  2. 2024LLM Chain Ensembles for Scalable and Accurate Data Annotationcited by 11
  3. 2026When Routing Collapses: On the Degenerate Convergence of LLM Routerscited by 10
  4. 2025TagRouter: Learning Route to LLMs through Tags for Open-Domain Text Generation Taskscited by 9
  5. 2025Learning Compact Representations of LLM Abilities via Item Response Theorycited by 9
  6. 2026Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learningcited by 9
  7. 2025Ensemble Learning for Large Language Models in Text and Code Generation: A Surveycited by 8
  8. 2026Task-Aware LLM Routing with Multi-Level Task-Profile-Guided Data Synthesis for Cold-Start Scenarioscited by 8
  9. 2024SelectLLM: Query-Aware Efficient Selection Algorithm for Large Language Modelscited by 7
  10. 2025LLM Routing with Dueling Feedbackcited by 6
  11. 2025PICE: A Semantic-Driven Progressive Inference System for LLM Serving in Cloud-Edge Networkscited by 5
  12. 2025Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniquescited by 5
  13. 2025Scoring, Reasoning, and Selecting the Best! Ensembling Large Language Models via a Peer-Review Processcited by 4
  14. 2026RACER: Risk-Aware Calibrated Efficient Routing for Large Language Modelscited by 4
  15. 2026GenRouter: Unified Workflow Routing for Agentic Image Generationcited by 4
  16. 2025MArgE: Meshing Argumentative Evidence from Multiple Large Language Models for Justifiable Claim Verificationcited by 3
  17. 2025CTTS: Collective Test-Time Scalingcited by 3
  18. 2026Opportunity Is Not Realizability: Selection-Valid Diagnostics for Multi-LLM Routingcited by 3
  19. 2025Synergistic Weak-Strong Collaboration by Aligning Preferencescited by 2
  20. 2025Balancing Information Accuracy and Response Timeliness in Networked LLMscited by 2
  21. 2025Beyond Fast and Slow: Cognitive-Inspired Elastic Reasoning for Large Language Modelscited by 2
  22. 2026Picking the Right Specialist: Attentive Neural Process-based Selection of Task-Specialized Models as Tools for Agentic Healthcare Systemscited by 2
  23. 2025Teaming LLMs to Detect and Mitigate Hallucinationscited by 1
  24. 2026RouteJudge: An Open Platform for Reproducible and Preference-Aware LLM Routingcited by 1
  25. 2025EMAFusion: A Self-Optimizing System for Seamless LLM Selection and Integrationcited by 0
  26. 2025A Scalable Multi-LLM Collaboration System with Retrieval-based Selection and Exploration-Exploitation-Driven Enhancementcited by 0
  27. 2026Learning to Assign Prediction Tasks to Agents with Capacity Constraintscited by 0
  28. 2026SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversationscited by 0
  29. 2026Mo' Models, Mo' Problems: How to best select model pools when designing Multi-Agent Systemscited by 0
  30. 2026Routing Without Embeddings: Fast And Interpretable Routing With Regular Expressionscited by 0
  31. 2026FlexRouter: Learning Complementary Model Sets for Flexible LLM Routingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning Agent Routing From Early Experience
  2. 2026LLM Routing as Reasoning: A MaxSAT View
  3. 2026Reasoning Is Not Free: Robust Adaptive Cost-Efficient Routing for LLM-as-a-Judge
  4. 2026Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agents
  5. 2025RADAR: Reasoning-Ability and Difficulty-Aware Routing for Reasoning LLMs
  6. 2025When to Reason: Semantic Router for vLLM
  7. 2026ODAR: Principled Adaptive Routing for LLM Reasoning via Active Inference
  8. 2026Measured Joules, Learned Routes: Learning to Route for Energy-Efficient LLM Serving