# Cost-Optimal LLM Routing with Limited User Feedback under User Satisfaction Guarantees

arXiv: https://arxiv.org/abs/2606.19376 · HTML: https://mlsanity.com/paper/2606.19376/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 29 references that are in the map.
- First posted on arXiv 12 Jun 2026.

## Builds on (most cited first, 25 of 25)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions](https://mlsanity.com/paper/1905.10044/) (2019)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [Crowdsourcing Multiple Choice Science Questions](https://mlsanity.com/paper/1707.06209/) (2017)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [A Comprehensive Overview of Large Language Models](https://mlsanity.com/paper/2307.06435/) (2023)
- [Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing](https://mlsanity.com/paper/2404.14618/) (2024)
- [From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference](https://mlsanity.com/paper/2310.03003/) (2023)
- [AutoMix: Automatically Mixing Language Models](https://mlsanity.com/paper/2310.12963/) (2023)
- [RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Models](https://mlsanity.com/paper/2409.19886/) (2024)
- [TensorOpera Router: A Multi-Model Router for Efficient LLM Inference](https://mlsanity.com/paper/2408.12320/) (2024)
- [ACPBench: Reasoning about Action, Change, and Planning](https://mlsanity.com/paper/2410.05669/) (2024)
- [CARROT: A Cost Aware Rate Optimal Router](https://mlsanity.com/paper/2502.03261/) (2025)
- [Causal LLM Routing: End-to-End Regret Minimization from Observational Data](https://mlsanity.com/paper/2505.16037/) (2025)
- [xRouter: Training Cost-Aware LLMs Orchestration System via Reinforcement Learning](https://mlsanity.com/paper/2510.08439/) (2025)
- [LLM DNA: Tracing Model Evolution via Functional Representations](https://mlsanity.com/paper/2509.24496/) (2025)
- [Breaking Feedback Loops in Recommender Systems with Causal Inference](https://mlsanity.com/paper/2207.01616/) (2022)
- [MESS+: Dynamically Learned Inference-Time LLM Routing in Model Zoos with Service Level Guarantees](https://mlsanity.com/paper/2505.19947/) (2025)
- [Lookahead Routing for Large Language Models](https://mlsanity.com/paper/2510.19506/) (2025)
- [PROTEUS: SLA-Aware Routing via Lagrangian RL for Multi-LLM Serving Systems](https://mlsanity.com/paper/2601.19402/) (2026)

## Built on by (most cited first, 3 of 3)

- [You Cannot Pick a Provider From the Price List: Market-Aware Routing for Open-Weight LLM Inference](https://mlsanity.com/paper/2609.37902/) (2026)
- [ROUTEAUDIT: Interaction-Aware Identification for Budgeted Multi-Verifier Routing](https://mlsanity.com/paper/2610.02808/) (2026)
- [Choosing LLMs for Strategic Agent Interactions](https://mlsanity.com/paper/2610.18354/) (2026)

## Abstract (arXiv)

Inference costs for large language model (LLM) applications are rapidly growing, driven by surging demand and rising infrastructure cost. Users expect high-quality responses, and in commercial settings this is formally codified in Service Level Agreements (SLAs), creating a fundamental tension between cost and quality. Recent progress on cost-aware LLM request routing has shown potential to resolve this tension, but existing approaches rely on complete feedback signals, offline training, extensive per-workload tuning, and most lack SLA guarantees or inference-time adaptivity. We introduce SLARouter, an online routing algorithm that learns a cost-optimal policy from the sparse, one-sided user feedback available in production systems. SLARouter provides theoretical guarantees for both cost optimality and strict SLA compliance. Experiments across a wide range of LLM benchmarks show that SLARouter satisfies SLA constraints without the need for per-benchmark tuning, reducing operating cost by up to 2.2x over existing baselines.