# ToolOrchestra: Elevating Intelligence via Efficient Model and Tool Orchestration

arXiv: https://arxiv.org/abs/2511.21689 · HTML: https://mlsanity.com/paper/2511.21689/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 51 references that are in the map.
- First posted on arXiv 26 Nov 2025.

## Builds on (most cited first, 30 of 46)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://mlsanity.com/paper/2503.09516/) (2025)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvement](https://mlsanity.com/paper/2409.12122/) (2024)
- [τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://mlsanity.com/paper/2506.07982/) (2025)
- [Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs](https://mlsanity.com/paper/2503.01743/) (2025)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [Humanity's Last Exam](https://mlsanity.com/paper/2501.14249/) (2025)
- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning](https://mlsanity.com/paper/2503.04697/) (2025)
- [ToolRL: Reward is All Tool Learning Needs](https://mlsanity.com/paper/2504.13958/) (2025)
- [Tool Learning with Foundation Models](https://mlsanity.com/paper/2304.08354/) (2023)
- [Small Language Models are the Future of Agentic AI](https://mlsanity.com/paper/2506.02153/) (2025)
- [Training Language Models to Reason Efficiently](https://mlsanity.com/paper/2502.04463/) (2025)
- [WebSailor: Navigating Super-human Reasoning for Web Agent](https://mlsanity.com/paper/2507.02592/) (2025)
- [ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL](https://mlsanity.com/paper/2402.19446/) (2024)
- [WebDancer: Towards Autonomous Information Seeking Agency](https://mlsanity.com/paper/2505.22648/) (2025)
- [ToRL: Scaling Tool-Integrated RL](https://mlsanity.com/paper/2503.23383/) (2025)
- [Llama-Nemotron: Efficient Reasoning Models](https://mlsanity.com/paper/2505.00949/) (2025)
- [RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning](https://mlsanity.com/paper/2410.02089/) (2024)
- [OpenMathInstruct-1: A 1.8 Million Math Instruction Tuning Dataset](https://mlsanity.com/paper/2402.10176/) (2024)

## Built on by (most cited first, 30 of 47)

- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [Code as Agent Harness](https://mlsanity.com/paper/2605.18747/) (2026)
- [AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration](https://mlsanity.com/paper/2602.03786/) (2026)
- [Toward Efficient Agents: Memory, Tool learning, and Planning](https://mlsanity.com/paper/2601.14192/) (2026)
- [Autogenesis: A Self-Evolving Agent Protocol](https://mlsanity.com/paper/2604.15034/) (2026)
- [PlanBench-XL: Evaluating Long-Horizon Planning of LLM Tool-Use Agents in Large-Scale Tool Ecosystems](https://mlsanity.com/paper/2606.22388/) (2026)
- [α3-Bench: A Unified Benchmark of Safety, Robustness, and Efficiency for LLM-Based UAV Agents over 6G Networks](https://mlsanity.com/paper/2601.03281/) (2026)
- [Decentralized Multi-Agent Systems with Shared Context](https://mlsanity.com/paper/2606.10662/) (2026)
- [Training AI Scientists to Replicate Research](https://mlsanity.com/paper/2608.13331/) (2026)
- [An Information Theoretic Perspective on Agentic System Design](https://mlsanity.com/paper/2512.21720/) (2025)
- [Ares: Adaptive Reasoning Effort Selection for Efficient LLM Agents](https://mlsanity.com/paper/2603.07915/) (2026)
- [Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Application](https://mlsanity.com/paper/2606.12191/) (2026)
- [Uno-Orchestra: Parsimonious Agent Routing via Selective Delegation](https://mlsanity.com/paper/2605.05007/) (2026)
- [Sparrow: Sparse Rollout for Stable and Efficient Long-context RL of Large Language Models](https://mlsanity.com/paper/2606.08446/) (2026)
- [PrefillShare: A Shared Prefill Module for KV Reuse in Multi-LLM Disaggregated Serving](https://mlsanity.com/paper/2602.12029/) (2026)
- [Agent Explorative Policy Optimization for Multimodal Agentic Reasoning](https://mlsanity.com/paper/2605.28774/) (2026)
- [APPO: Agentic Procedural Policy Optimization](https://mlsanity.com/paper/2606.12384/) (2026)
- [Experience as a Compass: Multi-agent RAG with Evolving Orchestration and Agent Prompts](https://mlsanity.com/paper/2604.00901/) (2026)
- [Agentic AI for Remote Sensing: Technical Challenges and Research Directions](https://mlsanity.com/paper/2604.24919/) (2026)
- [Orchestra-o1: Omnimodal Agent Orchestration](https://mlsanity.com/paper/2606.13707/) (2026)
- [Multi-Head Latent Control: A Unified Interface for LLM Agent Decision Making](https://mlsanity.com/paper/2607.14277/) (2026)
- [Speculate While You Reason: Teaching Agents to Predict Their Next Tool Call via Joint Agent-Speculator RL](https://mlsanity.com/paper/2607.25816/) (2026)
- [Architectural Implications of Agentic AI Workflows](https://mlsanity.com/paper/2608.04458/) (2026)
- [AGI Requires a Coordination Layer on Top of Pattern Repositories](https://mlsanity.com/paper/2512.05765/) (2025)
- [SUN: Shared Use of Next-token Prediction for Efficient Multi-LLM Disaggregated Serving](https://mlsanity.com/paper/2603.02599/) (2026)
- [Outcome-Aware Tool Selection for Semantic Routers: Latency-Constrained Learning Without LLM Inference](https://mlsanity.com/paper/2603.13426/) (2026)
- [LEMON: Learning Executable Multi-Agent Orchestration via Counterfactual Reinforcement Learning](https://mlsanity.com/paper/2605.14483/) (2026)
- [STAR-P'olyaMath: Multi-Agent Reasoning under Persistent Meta-Strategic Supervision](https://mlsanity.com/paper/2605.19338/) (2026)
- [Continual Model Routing in Evolving Model Hubs](https://mlsanity.com/paper/2605.28577/) (2026)
- [When Cloud Agents Meet Device Agents: Lessons from Hybrid Multi-Agent Systems](https://mlsanity.com/paper/2605.30102/) (2026)

## Abstract (arXiv)

Large language models are powerful generalists, yet solving deep and complex problems such as those of the Humanity's Last Exam (HLE) remains both conceptually challenging and computationally expensive. We show that small orchestrators managing other models and a variety of tools can both push the upper bound of intelligence and improve efficiency in solving difficult agentic tasks. We introduce ToolOrchestra, a method for training small orchestrators that coordinate intelligent tools. ToolOrchestra explicitly uses reinforcement learning with outcome-, efficiency-, and user-preference-aware rewards. Using ToolOrchestra, we produce Orchestrator, an 8B model that achieves higher accuracy at lower cost than previous tool-use agents while aligning with user preferences on which tools are to be used for a given query. On HLE, Orchestrator achieves a score of 37.1%, outperforming GPT-5 (35.1%) while being 2.5x more efficient. On tau2-Bench and FRAMES, Orchestrator surpasses GPT-5 by a wide margin while using only about 30% of the cost. Extensive analysis shows that Orchestrator achieves the best trade-off between performance and cost under multiple metrics, and generalizes robustly to unseen tools. These results demonstrate that composing diverse tools with a lightweight orchestration model is both more efficient and more effective than existing methods, paving the way for practical and scalable tool-augmented reasoning systems.