mlsanityData to 7 Oct 2026

Routing Should Pay for Itself: Sparse Supervision for Economical LLM Routing

arXiv 2609.37402 ↗First posted 29 Sep 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language model (LLM) routing reduces serving cost by assigning each query to an appropriate model while preserving response quality. Learning such a router, however, often requires executing multiple candidate models on historical queries to collect query--model quality feedback, creating a nontrivial supervision cost before deployment. Existing work largely focuses on serving-time efficiency, overlooking whether the resulting savings are sufficient to recover this upfront expenditure. We further observe that routing quality often saturates well before all query--model feedback is collected, suggesting that dense supervision can be economically over-provisioned. We propose SaveRouter, a sparse-supervision routing framework that selectively acquires informative model feedback and shares capability information across related queries, while retaining query-level refinement for fine-grained routing. We evaluate routing by jointly accounting for supervision expenditure and subsequent serving-time savings. Across four routing benchmarks, the main setting uses only about 33--41% of available training feedback while maintaining competitive or better routing quality, and reduces the break-even deployment volume by approximately 1.9--9.5 times compared with the fastest conventional router. Further analysis shows that acquiring more supervision is not always economically preferable: the supervision level that minimizes serving cost can differ from the one that achieves the earliest payback. Our code is publicly available at https://github.com/LAMDA-Model-Reuse/SaveRouter.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 31 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

27 of 31 references.

References · 27

  1. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  2. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  3. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  4. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
  5. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
  6. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 154
  7. 2023AutoMix: Automatically Mixing Language Modelscited by 116
  8. 2023Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modelingcited by 94
  9. 2024RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Modelscited by 84
  10. 2021Active Testing: Sample-Efficient Model Evaluationcited by 82
  11. 2025Universal Model Routing for Efficient LLM Inferencecited by 80
  12. 2023Efficient Benchmarking of Language Modelscited by 71
15 more
  1. 2024TensorOpera Router: A Multi-Model Router for Efficient LLM Inferencecited by 59
  2. 2024EmbedLLM: Learning Compact Representations of Large Language Modelscited by 46
  3. 2025IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theorycited by 40
  4. 2025OmniRouter: Budget and Performance Controllable Multi-LLM Routingcited by 35
  5. 2026LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routingcited by 35
  6. 2024Active Evaluation Acquisition for Efficient LLM Benchmarkingcited by 33
  7. 2025Beyond GPT-5: Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routingcited by 32
  8. 2025Causal LLM Routing: End-to-End Regret Minimization from Observational Datacited by 27
  9. 2025Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offscited by 13
  10. 2026MMR-Bench: A Comprehensive Benchmark for Multimodal LLM Routingcited by 13
  11. 2026When Routing Collapses: On the Degenerate Convergence of LLM Routerscited by 10
  12. 2025INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profilingcited by 8
  13. 2026Task-Aware LLM Routing with Multi-Level Task-Profile-Guided Data Synthesis for Cold-Start Scenarioscited by 8
  14. 2026From Sampled Outcomes to Capability Distributions: Rethinking Supervision for LLM Routingcited by 3
  15. 2026WISERouter: LLM Routing with Workload Budget Constraintcited by 2

Built on by

Not cited yet.