mlsanityData to 7 Oct 2026

PROTEUS: SLA-Aware Routing via Lagrangian RL for Multi-LLM Serving Systems

arXiv 2601.19402 ↗First posted 27 Jan 2026cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Production LLM deployments serve diverse workloads where cost and quality requirements vary by customer tier, time of day, and query criticality. Model serving systems accept latency SLOs directly. LLM routers do not. They force operators to tune parameters offline and guess what accuracy might result. The relationship between parameters and outcomes is indirect, non-monotonic, and dataset-dependent. Operators need to specify accuracy targets, not infer them from opaque settings. We present PROTEUS (Polymorphic Router for Operational Target Enforcement with Unified SLA), a router that accepts accuracy targets tau as runtime input. PROTEUS uses Lagrangian dual control. A learned dual variable lambda tracks constraint violations during training and conditions the policy network. This lets the router translate specified tau values into routing decisions that satisfy them. A single trained model serves the full accuracy spectrum without retraining.We evaluate on RouterBench (11 models, 405K queries) and SPROUT (14 models, 45K queries). PROTEUS achieves consistent floor compliance where accuracy meets or exceeds tau. The target-response correlation reaches 0.97 to 0.98. The closest baseline, OmniRouter, meets floors only 22% of the time despite also using Lagrangian optimization. PROTEUS operates across tau in [0.85, 0.95] from a single model. On RouterBench it achieves 90.1% accuracy, within 1.3% of oracle. On SPROUT it achieves 94.0% accuracy, within 4.6% of oracle. Cost savings reach 89.8% versus the best fixed model.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 19 of its 28 references that are in the map. First posted on arXiv 27 Jan 2026.

Builds on

19 of 28 references.

References · 19

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2021Convex optimizationcited by 2,479
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  5. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  6. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  7. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  8. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  9. 2018Reward Constrained Policy Optimizationcited by 397
  10. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
  11. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  12. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
7 more
  1. 2016Clipper: A Low-Latency Online Prediction Serving Systemcited by 191
  2. 2023AutoMix: Automatically Mixing Language Modelscited by 116
  3. 2025Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learningcited by 64
  4. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 58
  5. 2025OmniRouter: Budget and Performance Controllable Multi-LLM Routingcited by 35
  6. 2025CARROT: A Cost Aware Rate Optimal Routercited by 34
  7. 2025Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026ParetoBandit: Budget-Paced Adaptive Routing for Non-Stationary LLM Servingcited by 3
  2. 2026SeqRoute: Global Budget-Aware Sequential LLM Routing via Offline Reinforcement Learningcited by 2
  3. 2026The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Projectcited by 1
  4. 2026Cost-Optimal LLM Routing with Limited User Feedback under User Satisfaction Guaranteescited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026R2-Router: A New Paradigm for LLM Routing with Reasoning
  2. 2026Accuracy Is Speed: Towards Long-Context-Aware Routing for Distributed LLM Serving
  3. 2026From Sampled Outcomes to Capability Distributions: Rethinking Supervision for LLM Routing
  4. 2026RouteGuard: Certifying Routing Gain in LLM Multi-Agent Systems When Complementarity Is Not Enough
  5. 2026RouteJudge: An Open Platform for Reproducible and Preference-Aware LLM Routing
  6. 2026RouterWise: Joint Resource Allocation and Routing for Latency-Aware Multi-Model LLM Serving
  7. 2026Towards Fair and Comprehensive Evaluation of Routers in Collaborative LLM Systems
  8. 2026Online Linear Programming for Multi-Objective Routing in LLM Serving