# LLM Bandit: Cost-Efficient LLM Generation via Preference-Conditioned Dynamic Routing

arXiv: https://arxiv.org/abs/2502.02743 · HTML: https://mlsanity.com/paper/2502.02743/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2025.

## Built on by (most cited first, 30 of 34)

- [Universal Model Routing for Efficient LLM Inference](https://mlsanity.com/paper/2502.08773/) (2025)
- [Harnessing Multiple Large Language Models: A Survey on LLM Ensemble](https://mlsanity.com/paper/2502.18036/) (2025)
- [RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs](https://mlsanity.com/paper/2503.10657/) (2025)
- [Cost-Aware Contrastive Routing for LLMs](https://mlsanity.com/paper/2508.12491/) (2025)
- [Learning to Route LLMs from Bandit Feedback: One Policy, Many Trade-offs](https://mlsanity.com/paper/2510.07429/) (2025)
- [Rethinking Predictive Modeling for LLM Routing: When Simple kNN Beats Complex Learned Routers](https://mlsanity.com/paper/2505.12601/) (2025)
- [Survey: Multi-Armed Bandits Meet Large Language Models](https://mlsanity.com/paper/2505.13355/) (2025)
- [LLM Routing with Dueling Feedback](https://mlsanity.com/paper/2510.00841/) (2025)
- [ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outliers](https://mlsanity.com/paper/2510.09852/) (2025)
- [One Head, Many Models: Cross-Attention Routing for Cost-Aware LLM Selection](https://mlsanity.com/paper/2509.09782/) (2025)
- [Reward Model Routing in Alignment](https://mlsanity.com/paper/2510.02850/) (2025)
- [ParetoBandit: Budget-Paced Adaptive Routing for Non-Stationary LLM Serving](https://mlsanity.com/paper/2604.00136/) (2026)
- [Near-Optimal Online Deployment and Routing for Streaming LLMs](https://mlsanity.com/paper/2506.17254/) (2025)
- [GreenServ: Energy-Efficient Context-Aware Dynamic Routing for Multi-Model LLM Inference](https://mlsanity.com/paper/2601.17551/) (2026)
- [Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluations](https://mlsanity.com/paper/2601.22318/) (2026)
- [Towards Fair and Comprehensive Evaluation of Routers in Collaborative LLM Systems](https://mlsanity.com/paper/2602.11877/) (2026)
- [Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing](https://mlsanity.com/paper/2607.09015/) (2026)
- [Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model](https://mlsanity.com/paper/2608.13867/) (2026)
- [Resource-efficient Inference with Foundation Model Programs](https://mlsanity.com/paper/2504.07247/) (2025)
- [The Thinking Therapist: Training Large Language Models to Deliver Acceptance and Commitment Therapy using Supervised Fine-Tuning and Odds Ratio Policy Optimization](https://mlsanity.com/paper/2509.09712/) (2025)
- [Adapter-Augmented Bandits for Online Multi-Constrained Multi-Modal Inference Scheduling](https://mlsanity.com/paper/2603.06403/) (2026)
- [A Component-Based Survey of Interactions between Large Language Models and Multi-Armed Bandits](https://mlsanity.com/paper/2601.12945/) (2026)
- [Learning to Prompt: Improving Student Engagement with Adaptive LLM-based High-School Tutoring](https://mlsanity.com/paper/2606.20138/) (2026)
- [SCATE: Learning to Supervise Coding Agents for Cost-Effective Test Generation](https://mlsanity.com/paper/2607.08983/) (2026)
- [PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference](https://mlsanity.com/paper/2607.20327/) (2026)
- [Online Budget Allocation with Censored Semi-Bandit Feedback](https://mlsanity.com/paper/2508.05844/) (2025)
- [Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models](https://mlsanity.com/paper/2602.00485/) (2026)
- [KRONE: Scalable LLM-Augmented Log Anomaly Detection via Hierarchical Abstraction](https://mlsanity.com/paper/2602.07303/) (2026)
- [POLAR: Online Learning for LoRA Adapter Caching and Routing in Edge LLM Serving](https://mlsanity.com/paper/2604.16583/) (2026)
- [Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models](https://mlsanity.com/paper/2605.03426/) (2026)

## Abstract (arXiv)

The rapid advancement in large language models (LLMs) has brought forth a diverse range of models with varying capabilities that excel in different tasks and domains. However, selecting the optimal LLM for user queries often involves a challenging trade-off between accuracy and cost, a problem exacerbated by the diverse demands of individual queries. In this work, we present a novel framework that formulates the LLM selection process as a multi-armed bandit problem, enabling dynamic and intelligent routing of queries to the most appropriate model. Our approach incorporates a preference-conditioned dynamic routing mechanism, allowing users to specify their preferences at inference time, thereby offering a customizable balance between performance and cost. Additionally, our selection policy is designed to generalize to unseen LLMs, ensuring adaptability to new models as they emerge. Experimental results demonstrate that our method achieves significant improvements in both accuracy and cost-effectiveness across various LLM platforms, showcasing the potential of our framework to adaptively optimize LLM selection in real-world scenarios.