TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Modern enterprise agent deployments consist of a heterogeneous pool of large language models (LLMs) having diverse capabilities and cost. Existing model routing strategies optimize the quality-cost trade-off, while providing request-level static decisions. More recent solutions address agentic routing as a task-level selection with a serial verifier based router feedback loop. However, their fixed verifier suitable for homogeneous workloads may not generalize to heterogeneous batches of agentic tasks (example: coding, general conversational). Additionally, due to the verifier placement in the critical path of the loop, serving quality may be affected during multiple concurrent requests routing. To mitigate these issues, we present ORACLE. It is a concurrency-aware online routing mechanism that aligns adaptive routing with adaptive verification for feedback. ORACLE acts as a training-free drop-in 'feedback loop' on top of any model-selection policy to first classify the task type and then dynamically assigns a task-appropriate verifier. Further, we develop a delayed feedback strategy for concurrent requests that largely removes verifier latency from the dispatch critical path. We then present a post-routing dispatch scheduler, namely DISC. DISC reserves each task's peak KV footprint at admission and dispatches to an alternate backend when the reward gain from reduced wait exceeds the reward loss from lower accuracy. Extensive evaluation on SWE-bench, tau2-bench, and Terminal-Bench 2.0 shows that ORACLE improves the accuracy-cost frontier by up to 7 percentage points over state-of-the-art routing baselines, while ORACLE with DISC improves program throughput by up to 1.8x. To facilitate reproducibility and support future development, we open-source the code-base of ORACLE at https://github.com/ORACLE-org/ORACLE.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 33 of its 44 references that are in the map. First posted on arXiv 24 Jul 2026.
Builds on
33 of 44 references.
Closest ancestors · 2
Same direction, newest first
- 2026Agent-as-a-Router: Agentic Model Routing for Coding Tasks22 Jun
- 2024Agent-as-a-Judge: Evaluate Agents with Agents14 Oct
Other references · 31
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
- 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
- 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
- 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 421
- 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
- 2024JudgeBench: A Benchmark for Evaluating LLM-based Judgescited by 318
- 2024Llumnix: Dynamic Scheduling for Large Language Model Servingcited by 174
19 more
- 2013Online Learning under Delayed Feedbackcited by 141
- 2023AutoMix: Automatically Mixing Language Modelscited by 125
- 2023Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modelingcited by 95
- 2025MixLLM: Dynamic Routing in Mixed Large Language Modelscited by 59
- 2024MuxServe: Flexible Spatial-Temporal Multiplexing for Multiple LLM Servingcited by 52
- 2025Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systemscited by 51
- 2025Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifierscited by 45
- 2026LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routingcited by 38
- 2025Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooningcited by 24
- 2025Devstral: Fine-tuning Language Models for Coding Agent Applicationscited by 24
- 2026ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference Systemcited by 17
- 2026CONCUR: High-Throughput Agentic Batch Inference of LLM via Congestion-Based Concurrency Controlcited by 14
- 2026R2-Router: A New Paradigm for LLM Routing with Reasoningcited by 14
- 2026SWE-Router: Routing in Multi-turn Agentic Software Engineering Taskscited by 11
- 2026vLLM Semantic Router: Signal Driven Decision Routing for Mixture-of-Modality Modelscited by 6
- 2025Aragog: Just-in-Time Model Routing for Scalable Serving of Agentic Workflowscited by 5
- 2025Near-Optimal Online Deployment and Routing for Streaming LLMscited by 3
- 2026Robust KV Cache Management for LLM Serving under Output Token Length Uncertaintycited by 3
- 2026RankGuide: Tensor-Rank-Guided Routing and Steering for Efficient Reasoningcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026RSI-Router: Evolving Subtask-Level LLM Routing and Skills for Cost-Efficient Agents
- 2026Agentic Routing: The Harness-Native Data Flywheel
- 2026Learning Compositional Meta-Routing for Agentic Workflows: An Executable Benchmark
- 2026HM-ROUTER: Joint Model and Harness Routing for Agentic Systems
- 2026Symphony-Coord: Adaptive Routing for Multi-Agent LLM Systems
- 2026Runtime Burden Allocation for Structured LLM Routing in Agentic Expert Systems: A Full-Factorial Cross-Backend Methodology
- 2026CASTER: Breaking the Cost-Performance Barrier in Multi-Agent Orchestration via Context-Aware Strategy for Task Efficient Routing
- 2025Adaptive routing protocols for determining optimal paths in AI multi-agent systems: a priority- and learning-enhanced approach