mlsanityData to 14 Oct 2026

RACER: Risk-Aware Calibrated Efficient Routing for Large Language Models

arXiv 2603.06616 ↗First posted 20 Feb 2026cs.LG cs.AI math.STcited by 7

Not on the map. Why?

Abstract (arXiv)

Efficiently routing queries to the optimal large language model (LLM) is crucial for optimizing the cost-performance trade-off in multi-model systems. However, most existing routers rely on single-model selection, making them susceptible to misrouting. In this work, we formulate LLM routing as the α-VOR problem to minimize expected set size while controlling the misrouting risk, and propose a novel method -- RACER, extending base routers to output model sets that can be subsequently aggregated for improved output. In particular, RACER constructs nested model sets via augmented scoring and utilizes finite-sample concentration bounds to calibrate a threshold that allows for both variable set sizes and abstention. We theoretically prove that RACER achieves rigorous distribution-free risk control on unseen test data in a post-hoc and model-agnostic manner. Extensive experiments verify our theoretical guarantees and demonstrate that RACER consistently enhances downstream accuracy across a wide range of benchmarks.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 31 of its 39 references that are in the map. First posted on arXiv 20 Feb 2026.

Builds on

31 of 39 references.

References · 31

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  6. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,254
  7. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  8. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
  9. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  10. 2023MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Modelscited by 735
  11. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  12. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
19 more
  1. 2023Zephyr: Direct Distillation of LM Alignmentcited by 506
  2. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 449
  3. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
  4. 2022Conformal Risk Controlcited by 299
  5. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 238
  6. 2024More Agents Is All You Needcited by 180
  7. 2023Large Language Model Routing with Benchmark Datasetscited by 172
  8. 2023Conformal Language Modelingcited by 170
  9. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 165
  10. 2023Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Modelscited by 145
  11. 2023Conformal Prediction with Large Language Models for Multi-Choice Question Answeringcited by 139
  12. 2025Confidence Improves Self-Consistency in LLMscited by 113
  13. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  14. 2025Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learningcited by 72
  15. 2025BEST-Route: Adaptive LLM Routing with Test-Time Optimal Computecited by 67
  16. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 61
  17. 2024Harnessing the Power of Multiple Minds: Lessons Learned from LLM Routingcited by 46
  18. 2024Prune 'n Predict: Optimizing LLM Decision-making with Conformal Predictioncited by 18
  19. 2025A note on conditional PAC-efficient reasoning in large language model routingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026RELISH: LLM REgression with a Latent Iterative State Headcited by 2
  2. 2026Certify or Refuse: A Cross-Model Map for Selective Risk Control with Coverage Floors under Covariate Shiftcited by 0
  3. 2026SafeRestore: Detector-Relative Risk Certificates for Selective Industrial Image Restorationcited by 0
  4. 2026On the Pitfalls of Verbalized Confidence Priors for Calibrating Large Reasoning Modelscited by 0
  5. 2026Contingent Exposure Routing for Financial AI: Outage Risk and the Cost of Indivisible Decisionscited by 0
  6. 2026Do I Need the Cloud? Uncertainty-Aware Step-Level Handoff for Small Language Model Agentscited by 0
  7. 2026CopRoute: Dependence-Aware Conformal Routing for Language Modelscited by 0