mlsanityData to 7 Oct 2026

Switchcraft: AI Model Router for Agentic Tool Calling

arXiv 2605.07112 ↗First posted 8 May 2026cs.AI cs.MAcited by 3

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Agentic AI systems that invoke external tools are powerful but costly, leading developers to default to large models and overspend inference budgets. Model routing can mitigate this, but existing routers are designed for chat completion rather than tool use. We present Switchcraft, the first (to the best of our knowledge) model router optimized for agentic tool calling. Switchcraft operates inline, selecting the lowest-cost model subject to correctness. We construct an evaluation framework on five function-calling benchmarks and train a DistilBERT-based classifier, deployed under a latency budget. Switchcraft achieves 82.9% accuracy -- matching or exceeding the best individual model -- while reducing inference cost by 84%, saving over 3,600 per million queries. We find that larger models do not consistently outperform smaller ones on tool-use tasks, and that nominally cheaper models can incur higher total cost due to token-intensive reasoning. Our work enables cost-aware agentic AI deployment without sacrificing correctness.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 31 of its 41 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

31 of 41 references.

Closest ancestors · 4

Same direction, newest first

  1. 2026Explainable Model Routing for Agentic Workflows4 Apr
  2. 2025Agentic AI: A Comprehensive Survey of Architectures, Applications, and Future Directions29 Oct
  3. 2024xLAM: A Family of Large Action Models to Empower AI Agent Systems5 Sep
  4. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun

Other references · 27

  1. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  5. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  6. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 354
  7. 2024RouterBench: A Benchmark for Multi-LLM Routing Systemcited by 222
  8. 2023Large Language Model Routing with Benchmark Datasetscited by 166
  9. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 154
  10. 2023AutoMix: Automatically Mixing Language Modelscited by 116
  11. 2023Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modelingcited by 94
  12. 2024RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Modelscited by 84
15 more
  1. 2025Universal Model Routing for Efficient LLM Inferencecited by 80
  2. 2025BEST-Route: Adaptive LLM Routing with Test-Time Optimal Computecited by 63
  3. 2024TensorOpera Router: A Multi-Model Router for Efficient LLM Inferencecited by 59
  4. 2025IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theorycited by 40
  5. 2024MetaLLM: A High-performant and Cost-efficient Dynamic Framework for Wrapping LLMscited by 39
  6. 2025OmniRouter: Budget and Performance Controllable Multi-LLM Routingcited by 35
  7. 2026LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routingcited by 35
  8. 2025CARROT: A Cost Aware Rate Optimal Routercited by 34
  9. 2025ICL-Router: In-Context Learned Model Representations for LLM Routingcited by 26
  10. 2025RouterArena: An Open Platform for Comprehensive Comparison of LLM Routerscited by 19
  11. 2025CONFETTI: Conversational Function-Calling Evaluation Through Turn-Level Interactionscited by 16
  12. 2026R2-Router: A New Paradigm for LLM Routing with Reasoningcited by 13
  13. 2025IPR: Intelligent Prompt Routing with User-Controlled Quality-Cost Trade-offscited by 5
  14. 2023Validation of Modern JSON Schema: Formalization and Complexitycited by 2
  15. 2025RouteNator: A Router-Based Multi-Modal Architecture for Generating Synthetic Training Data for Function Calling LLMscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026The Replay Gap: Static Evaluation of Model Switching in LLM Agents Scores the Wrong Worldcited by 7
  2. 2026Agentic Routing: The Harness-Native Data Flywheelcited by 4
  3. 2026Tokengeist: Multi-Turn Attribution Tracing in Agentic Conversationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentPerfBench: A Benchmarking and Evaluation Suite for Inference Performance of Agentic LLMs
  2. 2026MAVEN: Improving Generalization in Agentic Tool Calling
  3. 2025AgentFlux: Decoupled Fine-Tuning & Inference for On-Device Agentic Systems
  4. 2025On Generalization in Agentic Tool Calling: CoreThink Agentic Reasoner and MAVEN Dataset
  5. 2026AgentRouter: Heterogeneous Model Routing for Cost-Optimal Multi-Step Agentic Workflows
  6. 2026Beyond State Machines: Executing Network Procedures with Agentic Tool-Calling Sequences
  7. 2026The Remarkable Effectiveness of Providing AI Agents with Natural Language Tools: A Replication Study Validating NLT Performance Across 14 Models
  8. 2026HM-ROUTER: Joint Model and Harness Routing for Agentic Systems