mlsanityData to 14 Oct 2026

Cost-Aware Contrastive Routing for LLMs

arXiv 2508.12491 ↗First posted 17 Aug 2025cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

We study cost-aware routing for large language models across diverse and dynamic pools of models. Existing approaches often overlook prompt-specific context, rely on expensive model profiling, assume a fixed set of experts, or use inefficient trial-and-error strategies. We introduce Cost-Spectrum Contrastive Routing (CSCR), a lightweight framework that maps both prompts and models into a shared embedding space to enable fast, cost-sensitive selection. CSCR uses compact, fast-to-compute logit footprints for open-source models and perplexity fingerprints for black-box APIs. A contrastive encoder is trained to favor the cheapest accurate expert within adaptive cost bands. At inference time, routing reduces to a single k-NN lookup via a FAISS index, requiring no retraining when the expert pool changes and enabling microsecond latency. Across multiple benchmarks, CSCR consistently outperforms baselines, improving the accuracy-cost tradeoff by up to 25%, while generalizing robustly to unseen LLMs and out-of-distribution prompts.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 74 of its 98 references that are in the map. First posted on arXiv 17 Aug 2025.

Builds on

74 of 98 references.

References · 74

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2020Scaling Laws for Neural Language Modelscited by 6,586
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  8. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  10. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  11. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  12. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
38 more
  1. 2024Mixtral of Expertscited by 1,823
  2. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  3. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  4. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,396
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  6. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  7. 2022Editing Models with Task Arithmeticcited by 1,120
  8. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  9. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  10. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  11. 2023TIES-Merging: Resolving Interference When Merging Modelscited by 818
  12. 2021BARTScore: Evaluating Generated Text as Text Generationcited by 722
  13. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  14. 2022Mixture-of-Experts with Expert Choice Routingcited by 623
  15. 2021Merging Models with Fisher-Weighted Averagingcited by 621
  16. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  17. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 564
  18. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
  19. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
  20. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  21. 2018Blockwise Parallel Decoding for Deep Autoregressive Modelscited by 323
  22. 2022Confident Adaptive Language Modelingcited by 310
  23. 2023Improving Text Embeddings with Large Language Modelscited by 303
  24. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 273
  25. 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 263
  26. 2022Unified Scaling Laws for Routed Language Modelscited by 253
  27. 2023AdaMerging: Adaptive Model Merging for Multi-Task Learningcited by 252
  28. 2023From Sparse to Soft Mixtures of Expertscited by 248
  29. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 238
  30. 2017Predict Responsibly: Improving Fairness and Accuracy by Learning to Defercited by 233
  31. 2023Recommender Systems in the Era of Large Language Models (LLMs)cited by 189
  32. 2023Large Language Model Routing with Benchmark Datasetscited by 172
  33. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 165
  34. 2023Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Modelscited by 145
  35. 2023Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoningcited by 143
  36. 2023SpecTr: Fast Speculative Decoding via Optimal Transportcited by 129
  37. 2023AutoMix: Automatically Mixing Language Modelscited by 125
  38. 2024Language Model Cascades: Token-level uncertainty and beyondcited by 121

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 13.

16 papers.

Sort:

By area:

  1. 2025Multi-Crit: Benchmarking Multimodal Judges on Pluralistic Criteria-Followingcited by 15
  2. 2026R2-Router: A New Paradigm for LLM Routing with Reasoningcited by 14
  3. 2026SWE-Router: Routing in Multi-turn Agentic Software Engineering Taskscited by 11
  4. 2025Explore Data Left Behind in Reinforcement Learning for Reasoning Language Modelscited by 7
  5. 2026ParetoBandit: Budget-Paced Adaptive Routing for Non-Stationary LLM Servingcited by 4
  6. 2026Not All Errors Are Equal: Consequence-Aware Reasoning Compute Allocationcited by 3
  7. 2026Algorithm for Contextual Queueing Bandits with Rate-Optimal Queue Length Regretcited by 2
  8. 2025Catalog-Native LLM: Speaking Item-ID Dialect with Less Entanglement for Recommendationcited by 1
  9. 2026SecureRouter: Encrypted Routing for Efficient Secure Inferencecited by 1
  10. 2026Privacy-Preserving LLMs Routingcited by 1
  11. 2026Capability Self-Assessment in Large Language Modelscited by 1
  12. 2026What Should a Skill Remember? Quality--Cost Trade-offs in Cost-Aware Skill Rewriting for Language Model Agentscited by 1
4 more
  1. 2026The Capability Frontier: Benchmarks Miss 82% of Model Performancecited by 1
  2. 2026n-Musketeers: Reinforcement Learning Shapes Collaboration Among Language Modelscited by 0
  3. 2026Quantum Circuit Vision: Cost-Aware Evaluation of Visual AI Agents for Quantum Code Generationcited by 0
  4. 2026COMPAS: Difficulty-Aware Joint Search for Optimizing Code Generationcited by 0