mlsanityData to 14 Oct 2026

From Sampled Outcomes to Capability Distributions: Rethinking Supervision for LLM Routing

arXiv 2606.06924 ↗First posted 5 Jun 2026cs.LGcited by 3

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Existing LLM routing methods often construct supervision from a single sampled response for each query--model pair. Because LLM generation is stochastic, however, such an observation can be an unstable estimate of model capability: semantically equivalent query formulations and repeated decoding may yield different scores and even different model preferences. We show that this instability can further propagate from routing labels to learned routing policies. To address this issue, we propose DARS (Distribution-Aware Routing Supervision), which estimates query-level model capability from repeated observations spanning semantics-preserving query rewrites and stochastic decoding. DARS summarizes expected quality, expected cost, and performance variability to construct risk-aware supervision without changing the downstream router architecture. Experiments across diverse tasks and routing methods show that DARS generally improves routing utility and cost--quality trade-offs over single-shot supervision. Further analyses show that its benefits persist under moderate sampling budgets and different decoding temperatures. These results suggest that reliable LLM routing should move beyond individual sampled outcomes and instead model query-level capability distributions.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 27 of its 48 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

27 of 48 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Rethinking Predictive Modeling for LLM Routing: When Simple kNN Beats Complex Learned Routers19 May

Other references · 26

  1. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  2. 2023Let's Verify Step by Stepcited by 3,789
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  4. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  5. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  6. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  7. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
  8. 2023PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Promptscited by 226
  9. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 165
  10. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 143
  11. 2025BEST-Route: Adaptive LLM Routing with Test-Time Optimal Computecited by 67
  12. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 61
14 more
  1. 2024TensorOpera Router: A Multi-Model Router for Efficient LLM Inferencecited by 60
  2. 2023EcoAssistant: Using LLM Assistant More Affordably and Accuratelycited by 49
  3. 2024Eagle: Efficient Training-Free Router for Multi-LLM Inferencecited by 35
  4. 2025OmniRouter: Budget and Performance Controllable Multi-LLM Routingcited by 35
  5. 2025Beyond GPT-5: Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routingcited by 34
  6. 2024Optimising Calls to Large Language Models with Uncertainty-Based Two-Tier Selectioncited by 33
  7. 2025Causal LLM Routing: End-to-End Regret Minimization from Observational Datacited by 28
  8. 2023Sensitivity and Robustness of Large Language Models to Prompt Template in Japanese Text Classification Taskscited by 25
  9. 2026MMR-Bench: A Comprehensive Benchmark for Multimodal LLM Routingcited by 13
  10. 2025Life-Cycle Routing Vulnerabilities of LLM Routercited by 11
  11. 2025How Robust Are Router-LLMs? Analysis of the Fragility of LLM Routing Capabilitiescited by 11
  12. 2026When Routing Collapses: On the Degenerate Convergence of LLM Routerscited by 10
  13. 2025Quality-of-Service Aware LLM Routing for Edge Computing with Multiple Expertscited by 7
  14. 2025Dynamic LLM Routing and Selection based on User Preferences: Balancing Performance, Cost, and Ethicscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026RouteJudge: An Open Platform for Reproducible and Preference-Aware LLM Routingcited by 1
  2. 2026Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routingcited by 0
  3. 2026Routing Should Pay for Itself: Sparse Supervision for Economical LLM Routingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning to Route LLMs from Implicit Cost-Performance Preferences via Meta-Learning
  2. 2025LLM Routing with Dueling Feedback
  3. 2025Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs
  4. 2026R2-Router: A New Paradigm for LLM Routing with Reasoning
  5. 2026LLM Router: Rethinking Routing with Prefill Activations
  6. 2026Drift-Aware LLM Routing with Sparse Contexts and Shared Budgets
  7. 2026TwinRouterBench: Fast Static and Live Dynamic Evaluation for Realistic Agentic LLM Routing
  8. 2026RouteNLP: Closed-Loop LLM Routing with Conformal Cascading and Distillation Co-Optimization