mlsanityData to 7 Oct 2026

Eagle: Efficient Training-Free Router for Multi-LLM Inference

arXiv 2409.15518 ↗First posted 23 Sep 2024cs.LGcited by 34

Not on the map. Why?

Abstract (arXiv)

The proliferation of Large Language Models (LLMs) with varying capabilities and costs has created a need for efficient model selection in AI systems. LLM routers address this need by dynamically choosing the most suitable model for a given query based on task requirements and budget constraints. However, existing routers face challenges in scalability and real-time adaptation, particularly in high-volume online environments. We present Eagle, a novel LLM routing approach that combines global and local ELO ranking modules to overcome these limitations. By evaluating both general and specialized LLM abilities, Eagle provides a scalable, training-free solution that enhances model selection quality while reducing computational overhead. Our experiments across multiple datasets show Eagle consistently outperforms baseline methods, with improvements of up to 23.52 percent in Area Under Curve (AUC) scores. Moreover, Eagle demonstrates remarkable efficiency, requiring only 1/20 of baseline methods' time for initialization and 100 to 200 times faster incremental updates in online scenarios, making it well-suited for dynamic, high-volume online serving environments.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 18 of its 23 references that are in the map. First posted on arXiv 23 Sep 2024.

Builds on

18 of 23 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  10. 2021Program Synthesis with Large Language Modelscited by 3,509
  11. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  12. 2023Qwen Technical Reportcited by 3,273
6 more
  1. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
  2. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  3. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  4. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
  5. 2024Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLMcited by 26
  6. 2024Plato: Plan to Efficiently Decode for Large Language Model Inferencecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 17; 2026: 16.

34 papers.

Sort:

By area:

  1. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  2. 2025Universal Model Routing for Efficient LLM Inferencecited by 80
  3. 2025Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in Large Language Model-Based Systemscited by 18
  4. 2025Online Multi-LLM Selection via Contextual Bandits under Unstructured Context Evolutioncited by 18
  5. 2026Dynamic Model Routing and Cascading for Efficient LLM Inference: A Surveycited by 16
  6. 2026When Routing Collapses: On the Degenerate Convergence of LLM Routerscited by 10
  7. 2025Arch-Router: Aligning LLM Routing with Human Preferencescited by 9
  8. 2025C3PO: Optimized Large Language Model Cascades with Probabilistic Cost Constraints for Reasoningcited by 8
  9. 2024Bench-CoE: a Framework for Collaboration of Experts from Benchmarkcited by 7
  10. 2025Rethinking Predictive Modeling for LLM Routing: When Simple kNN Beats Complex Learned Routerscited by 7
  11. 2025Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Expertscited by 7
  12. 2025LLM Routing with Dueling Feedbackcited by 6
22 more
  1. 2026RoboRouter: Training-Free Policy Routing for Robotic Manipulationcited by 6
  2. 2025ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outlierscited by 5
  3. 2025One Head, Many Models: Cross-Attention Routing for Cost-Aware LLM Selectioncited by 4
  4. 2025LLMRank: Understanding LLM Strengths for Model Routingcited by 4
  5. 2026Route to Rome Attack: Directing LLM Routers to Expensive Models via Adversarial Suffix Optimizationcited by 4
  6. 2026Hera: Learning Long-Horizon Coordination for Device-Cloud Collaborative LLM Agentscited by 3
  7. 2026From Sampled Outcomes to Capability Distributions: Rethinking Supervision for LLM Routingcited by 3
  8. 2025Dynamic Optimizations of LLM Ensembles with Two-Stage Reinforcement Learning Agentscited by 2
  9. 2026Towards Fair and Comprehensive Evaluation of Routers in Collaborative LLM Systemscited by 2
  10. 2026Tandem: Riding Together with Large and Small Language Models for Efficient Reasoningcited by 2
  11. 2026Continual Model Routing in Evolving Model Hubscited by 1
  12. 2026RouteJudge: An Open Platform for Reproducible and Preference-Aware LLM Routingcited by 1
  13. 2026LLMVisor: A Real-Time Latency Attribution Model for Multi-Tenant LLM Servingcited by 1
  14. 2025EMAFusion: A Self-Optimizing System for Seamless LLM Selection and Integrationcited by 0
  15. 2025CoE-Ops: Collaboration of LLM-based Experts for AIOps Question-Answeringcited by 0
  16. 2025PRISM-Consult: A Panel-of-Experts Architecture for Clinician-Aligned Diagnosiscited by 0
  17. 2025CoLM: Collaborative Large Models via A Client-Server Paradigmcited by 0
  18. 2026Ensemble Self-Training for Unsupervised Machine Translationcited by 0
  19. 2026ReLope: KL-Regularized LoRA Probes for Multimodal LLM Routingcited by 0
  20. 2026The Routing Plateau: Understanding the Accuracy Limits of LLM Routerscited by 0
  21. 2026Training-Free versus Training-Based Intent Classification in LLMs: Accuracy, Robustness, and Failure Modescited by 0
  22. 2026TokenRouter: Efficient Serving System for Token-Level LLM Routingcited by 0