mlsanityData to 14 Oct 2026

Beyond GPT-5: Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routing

arXiv 2508.12631 ↗First posted 18 Aug 2025cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

Balancing performance and efficiency is a central challenge in large language model (LLM) advancement. GPT-5 addresses this with test-time routing, dynamically assigning queries to either an efficient or a high-capacity model during inference. In this work, we present Avengers-Pro, a test-time routing framework that ensembles LLMs of varying capacities and efficiencies, providing a unified solution for all performance-efficiency tradeoffs. The Avengers-Pro embeds and clusters incoming queries, then routes each to the most suitable model based on a performance-efficiency score. Across 6 challenging benchmarks and 8 leading models -- including GPT-5-medium, Gemini-2.5-pro, and Claude-opus-4.1 -- Avengers-Pro achieves state-of-the-art results: by varying a performance-efficiency trade-off parameter, it can surpass the strongest single model (GPT-5-medium) by +7% in average accuracy. Moreover, it can match the average accuracy of the strongest single model at 27% lower cost, and reach ~90% of that performance at 63% lower cost. Last but not least, it achieves a Pareto frontier, consistently yielding the highest accuracy for any given cost, and the lowest cost for any given accuracy, among all single models. Code is available at https://github.com/ZhangYiqun018/AvengersPro.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 26 of its 36 references that are in the map. First posted on arXiv 18 Aug 2025.

Builds on

26 of 36 references.

References · 26

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  5. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  6. 2023Large Language Modelscited by 780
  7. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  8. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
  9. 2025Humanity's Last Examcited by 491
  10. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
  11. 2024Measuring short-form factuality in large language modelscited by 373
  12. 2024Token-Budget-Aware LLM Reasoningcited by 247
14 more
  1. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 238
  2. 2023Large Language Model Routing with Benchmark Datasetscited by 172
  3. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 165
  4. 2023Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Modelscited by 145
  5. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectivenesscited by 119
  6. 2024ARC Prize 2024: Technical Reportcited by 104
  7. 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chainscited by 95
  8. 2024RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Modelscited by 88
  9. 2024Learning to Decode Collaboratively with Multiple Language Modelscited by 62
  10. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 61
  11. 2024Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Modelscited by 44
  12. 2025The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giantscited by 38
  13. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  14. 2025Decouple and Orthogonalize: A Data-Free Framework for LoRA Mergingcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 24.

34 papers.

Sort:

By area:

  1. 2026LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routingcited by 38
  2. 2026Dynamic Model Routing and Cascading for Efficient LLM Inference: A Surveycited by 19
  3. 2025Intelligence per Watt: Measuring Intelligence Efficiency of Local AIcited by 14
  4. 2025Learning Compact Representations of LLM Abilities via Item Response Theorycited by 10
  5. 2026When Routing Collapses: On the Degenerate Convergence of LLM Routerscited by 10
  6. 2025VL-RouterBench: A Benchmark for Vision-Language Model Routingcited by 8
  7. 2025LLM Routing with Dueling Feedbackcited by 7
  8. 2026V0: A Generalist Value Model for Any Policy at State Zerocited by 7
  9. 2025RADAR: Reasoning-Ability and Difficulty-Aware Routing for Reasoning LLMscited by 6
  10. 2025On the Provable Performance Guarantee of Efficient Reasoning Modelscited by 6
  11. 2025ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outlierscited by 6
  12. 2026Mid-Think: Training-Free Intermediate-Budget Reasoning via Token-Level Triggerscited by 6
22 more
  1. 2026Select-then-Solve: Paradigm Routing as Inference-Time Optimization for LLM Agentscited by 5
  2. 2026Route to Rome Attack: Directing LLM Routers to Expensive Models via Adversarial Suffix Optimizationcited by 5
  3. 2025Language Models and Logic Programs for Trustworthy Tax Reasoningcited by 4
  4. 2026Opportunity Is Not Realizability: Selection-Valid Diagnostics for Multi-LLM Routingcited by 4
  5. 2025Towards Generalized Routing: Model and Agent Orchestration for Adaptive and Efficient Inferencecited by 3
  6. 2026Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluationscited by 3
  7. 2026From GPT-3 to GPT-5: Mapping their capabilities, scope, limitations, and consequencescited by 3
  8. 2026From Sampled Outcomes to Capability Distributions: Rethinking Supervision for LLM Routingcited by 3
  9. 2026HyDRA: Hybrid Dynamic Routing Architecture for Heterogeneous LLM Poolscited by 1
  10. 2026The Routing Plateau: Understanding the Accuracy Limits of LLM Routerscited by 1
  11. 2026RouteJudge: An Open Platform for Reproducible and Preference-Aware LLM Routingcited by 1
  12. 2026RSI-Router: Evolving Subtask-Level LLM Routing and Skills for Cost-Efficient Agentscited by 1
  13. 2026Dynamic LLM Routers are Often Misguidedcited by 1
  14. 2025The ORCA Benchmark: Evaluating Real-World Calculation Accuracy in Large Language Modelscited by 0
  15. 2026Learning to Route LLMs from Implicit Cost-Performance Preferences via Meta-Learningcited by 0
  16. 2026RLCascadeRouter: Quality-Estimator-Free Cascade Routing via Reinforcement Learningcited by 0
  17. 2026AIBuildAI-2.5: Efficient Autonomous AI Model Development Through LLM-Guided Tree Searchcited by 0
  18. 2026HM-ROUTER: Joint Model and Harness Routing for Agentic Systemscited by 0
  19. 2026Routing Without Embeddings: Fast And Interpretable Routing With Regular Expressionscited by 0
  20. 2026Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routingcited by 0
  21. 2026Routing Should Pay for Itself: Sparse Supervision for Economical LLM Routingcited by 0
  22. 2026Embedding Geometry Shapes LLM Routing: Tracing Failure Modes and Improving Generalizationcited by 0