MetaRoute-Bench: Evaluating Meta-Decision Policies for Agentic Workflow Routing

arXiv 2608.00107 ↗First posted 31 Jul 2026cs.LG cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Agentic systems must repeatedly decide whether to answer directly, decompose a task, invoke a tool, execute code, delegate to a specialist, verify an intermediate result, or recover from failure. These meta-decisions affect not only task success but also operating cost and latency, yet they are often embedded inside an orchestration framework and evaluated only through aggregate task accuracy. We present MetaRoute-Bench, an open, inspectable framework for comparing meta-decision policies under a shared execution model. The initial benchmark contains 180 synthetic task profiles spanning data analysis, research, and document processing, eight routing policies, and 30 paired random seeds. Across 43,200 traces, a task-aware compositional policy achieves 79.4% success compared with 76.7% for a strong workload-specific static policy, 67.4% for one-shot task routing, and 52.9% for direct answering. Relative to the static policy, this is a 2.7 percentage-point improvement with paired 95% CI of plus or minus 2.0 points, at 4.7% higher mean cost and 6.4% higher latency. Ablations show the largest losses when route composition is restricted to one operation and when verification is removed. These results are generated by a seeded offline execution model rather than a live deployment; accordingly, the primary contribution is a reproducible evaluation method and an analysis of routing-policy tradeoffs, not evidence of production effectiveness. We release task generation, policies, traces, tests, and analysis artifacts to support live-system validation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 10 of its 23 references that are in the map. First posted on arXiv 31 Jul 2026.

Builds on

10 of 23 references.

Closest ancestors · 2

Same direction, newest first

2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun
2023AgentBench: Evaluating LLMs as Agents7 Aug

Other references · 8

2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
2023An LLM Compiler for Parallel Function Callingcited by 179
2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 124
2025CP-Router: An Uncertainty-Aware Router Between LLM and LRMcited by 20

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way