mlsanityData to 14 Oct 2026

RLCascadeRouter: Quality-Estimator-Free Cascade Routing via Reinforcement Learning

arXiv 2608.15817 ↗First posted 16 Aug 2026cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

The growing ecosystem of large language models (LLMs) offers huge potential to optimize performance-cost trade-offs. However, their heterogeneous capabilities and inference costs make efficiently routing queries a significant challenge. Existing paradigms are inflexible: one-shot routers commit before observing responses, whereas conventional cascades stop adaptively but follow a fixed model order. Cascade routing removes both restrictions by reconsidering whether to stop or invoke another model after each response. Current methods use a predict-then-optimize pipeline estimating response quality and future model utility. However, prediction loss for quality or utility is not equivalent to routing-decision loss. A lower prediction error does not necessarily yield a better action; a small boundary-crossing error can reverse a ``stop'' or model-selection decision. Therefore, we propose RLCascadeRouter, a quality-estimator-free framework that formulates cascade routing as a Markov decision process with actions comprising ``stop'' and model selection. It uses trajectory returns and advantages to directly optimize the performance-cost objective. Its Cascade Policy Network models candidate complementarity for model selection and remaining-action value for stopping, eliminating independent post-hoc response-quality estimators. Evaluated across ten LLMRouterBench benchmarks with thirteen LLMs, RLCascadeRouter outperforms strong baselines and achieves superior performance-cost trade-offs. It incorporates unseen models without retraining, and ablation studies validate both policy components.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 26 of its 27 references that are in the map. First posted on arXiv 16 Aug 2026.

Builds on

26 of 27 references.

References · 26

  1. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
  2. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  3. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  4. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
  5. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 238
  6. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 165
  7. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 143
  8. 2023AutoMix: Automatically Mixing Language Modelscited by 125
  9. 2024Language Model Cascades: Token-level uncertainty and beyondcited by 121
  10. 2024RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Modelscited by 88
  11. 2023Exploring the Benefits of Training Expert Language Models over Instruction Tuningcited by 80
  12. 2024A Unified Approach to Routing and Cascading for LLMscited by 80
14 more
  1. 2025Router-R1: Teaching LLMs Multi-Round Routing and Aggregation via Reinforcement Learningcited by 72
  2. 2024Learning to Route LLMs with Confidence Tokenscited by 67
  3. 2025RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMscited by 61
  4. 2024EmbedLLM: Learning Compact Representations of Large Language Modelscited by 48
  5. 2024MetaLLM: A High-performant and Cost-efficient Dynamic Framework for Wrapping LLMscited by 41
  6. 2026LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routingcited by 38
  7. 2025Beyond GPT-5: Making LLMs Cheaper and Better via Performance-Efficiency Optimized Routingcited by 34
  8. 2025Adaptive LLM Routing under Budget Constraintscited by 24
  9. 2025Doing More with Less: A Survey on Routing Strategies for Resource Optimisation in Large Language Model-Based Systemscited by 20
  10. 2026Dynamic Model Routing and Cascading for Efficient LLM Inference: A Surveycited by 19
  11. 2024OptLLM: Optimal Assignment of Queries to Large Language Modelscited by 16
  12. 2024MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Designcited by 16
  13. 2025Route-and-Reason: Scaling Large Language Model Reasoning with Reinforced Model Routercited by 13
  14. 2024Efficient Contextual LLM Cascades through Budget-Constrained Policy Learningcited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025HierRouter: Coordinated Routing of Specialized Large Language Models via Reinforcement Learning
  2. 2026ReCal: Reward Calibration for RL-based LLM Routing
  3. 2026Models Under SCOPE: Scalable and Controllable Routing via Pre-hoc Reasoning
  4. 2026Signed Rescue Routing: Harm-Aware Cascades for Efficient LLM Inference
  5. 2025SATER: A Self-Aware and Token-Efficient Approach to Routing and Cascading
  6. 2026RouteWeaver: Weaving Mode Selection and Execution into Unified LLM Routing
  7. 2024PickLLM: Context-Aware RL-Assisted Large Language Model Routing
  8. 2025Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs