# SATER: A Self-Aware and Token-Efficient Approach to Routing and Cascading

arXiv: https://arxiv.org/abs/2510.05164 · HTML: https://mlsanity.com/paper/2510.05164/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 4 Oct 2025.

## Builds on (most cited first, 19 of 19)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing](https://mlsanity.com/paper/2404.14618/) (2024)
- [O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning](https://mlsanity.com/paper/2501.12570/) (2025)
- [Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMs](https://mlsanity.com/paper/2406.15927/) (2024)
- [ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning](https://mlsanity.com/paper/2002.04326/) (2020)
- [Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoning](https://mlsanity.com/paper/2310.03094/) (2023)
- [Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedback](https://mlsanity.com/paper/2403.18349/) (2024)
- [The Wisdom of Hindsight Makes Language Models Better Instruction Followers](https://mlsanity.com/paper/2302.05206/) (2023)
- [TensorOpera Router: A Multi-Model Router for Efficient LLM Inference](https://mlsanity.com/paper/2408.12320/) (2024)
- [Optimising Calls to Large Language Models with Uncertainty-Based Two-Tier Selection](https://mlsanity.com/paper/2405.02134/) (2024)

## Built on by (most cited first, 7 of 7)

- [Dynamic Model Routing and Cascading for Efficient LLM Inference: A Survey](https://mlsanity.com/paper/2603.04445/) (2026)
- [CascadeDebate: Multi-Agent Deliberation for Cost-Aware LLM Cascades](https://mlsanity.com/paper/2604.12262/) (2026)
- [On Time, Within Budget: Constraint-Driven Online Resource Allocation for Agentic Workflows](https://mlsanity.com/paper/2605.06110/) (2026)
- [From Positionwise Confidence to Prefix Scheduling: Verifier Skipping in Speculative Decoding](https://mlsanity.com/paper/2608.14787/) (2026)
- [Are These Modules Worth Their Cost? A Paradigm-Level Accuracy-Cost Analysis of In-context Learning Text-to-SQL](https://mlsanity.com/paper/2608.28432/) (2026)
- [COMED: The Missing Middle Between Routing and Collaboration in Multi-LLM Inference](https://mlsanity.com/paper/2609.26913/) (2026)
- [When Faster Decisions Do Not Make Better Agents: An Empirical Study of Jev in Agent Workflows](https://mlsanity.com/paper/2610.13092/) (2026)

## Abstract (arXiv)

Large language models (LLMs) demonstrate remarkable performance across diverse tasks, yet their effectiveness frequently depends on costly commercial APIs or cloud services. Model selection thus entails a critical trade-off between performance and cost: high-performing LLMs typically incur substantial expenses, whereas budget-friendly small language models (SLMs) are constrained by limited capabilities. Current research primarily proposes two routing strategies: pre-generation routing and cascade routing. Both approaches have distinct characteristics, with cascade routing typically offering superior cost-effectiveness and accuracy despite its higher latency. To further address the limitations of both approaches, we introduce SATER, a dual-mode compatible approach that fine-tunes models through shortest-response preference optimization and a confidence-aware rejection mechanism. SATER significantly reduces redundant outputs and response times, while improving both the performance of pre-generation routing and the efficiency of cascade routing. Experiments across three SLMs and six datasets, varying in type and complexity, demonstrate that SATER achieves comparable performance while consistently reducing computational costs by over 50% and cascade latency by over 80%.