Learning Compositional Meta-Routing for Agentic Workflows: An Executable Benchmark
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Agentic systems must decide not only what answer to produce, but which reasoning and execution operations should precede it. A controller may answer directly, decompose a request, retrieve evidence, execute code, delegate to a specialist, or verify an intermediate result. Existing routing work largely selects model endpoints, retrieval depth, or tools in isolation. We introduce an executable benchmark and a budget-aware meta-router that composes heterogeneous operations from raw task text. The benchmark contains 216 training, 72 development, 108 held-out test, and 108 locked lexical-shift challenge tasks across data analysis, frozen-corpus research, and document processing. Outcomes are machine checked after operations execute. Independent regularized logistic heads predict operation probabilities from word and character features, are temperature-scaled on development data, and are greedily composed under route-cost and action-count budgets. On the held-out test, the learned policy achieves 100% success versus 93.5% for strong static and fixed workflows, with 43% lower cost than the static policy; a matched learned one-shot router reaches 56.5%. On the untouched challenge split, learned success falls to 75.9% and trails static routing at 93.5%, while remaining 49% cheaper and exceeding one-shot routing by 34.3 points. The gap identifies lexical generalization, rather than route execution, as the principal limitation. These results establish a reproducible testbed and a bounded proof of concept, not evidence of live-LLM performance.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 14 of its 20 references that are in the map. First posted on arXiv 31 Jul 2026.
Builds on
14 of 20 references.
Closest ancestors · 1
Same direction, newest first
Other references · 13
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
- 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
- 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 437
- 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 272
- 2023An LLM Compiler for Parallel Function Callingcited by 170
- 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 121
- 2025ICL-Router: In-Context Learned Model Representations for LLM Routingcited by 26
- 2025CP-Router: An Uncertainty-Aware Router Between LLM and LRMcited by 19
- 2025SPIRAL: Symbolic LLM Planning via Grounded and Reflective Searchcited by 7
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026MetaRoute-Bench: Evaluating Meta-Decision Policies for Agentic Workflow Routing
- 2026Learning to Compose for Cross-domain Agentic Workflow Generation
- 2026AgentRouter: Heterogeneous Model Routing for Cost-Optimal Multi-Step Agentic Workflows
- 2026Explainable Model Routing for Agentic Workflows
- 2026AgentXRay: White-Boxing Agentic Systems via Workflow Reconstruction
- 2025AgentCompass: Towards Reliable Evaluation of Agentic Workflows in Production
- 2026Workflow-Localized Mechanism Learning: Attribution-Guided Repair and Knowledge Reuse for Structured Agent Skills
- 2026Agent-as-a-Router: Agentic Model Routing for Coding Tasks