mlsanityData to 7 Oct 2026

Scaling Enterprise Agent Routing: Degradation, Diagnosis, and Recovery

arXiv 2606.17519 ↗First posted 16 Jun 2026cs.CL cs.AIcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Production LLM assistants route user requests to growing libraries of specialized tools, but how does routing accuracy degrade as the catalog scales? We study single-step routing on a 110-agent, 584-tool catalog from a deployed enterprise productivity assistant, evaluating three frontier models from 10 to 110 agents. Routing F1 on under-specified requests drops 16--23 percentage points across models. An oracle analysis decomposes the degradation into a retrieval gap (the model cannot surface the right tool) and a confusion gap (even with perfect retrieval, the oracle ceiling drops 10pp). Embedding-based shortlisting recovers +10--11pp F1 at full scale across all three models and two providers. A production annotation study (1,435 human-labeled utterances, three annotators) confirms the recovery on real traffic at +10--17pp despite 10--15pp lower absolute performance.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 15 of its 26 references that are in the map. First posted on arXiv 16 Jun 2026.

Builds on

15 of 26 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Towards a Science of Scaling Agent Systems9 Dec

Other references · 14

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  2. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  3. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  4. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  5. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 238
  6. 2024APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasetscited by 188
  7. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 121
  8. 2024ToolGen: Unified Tool Retrieval and Calling via Generationcited by 72
  9. 2024Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmarkcited by 62
  10. 2026SkillRouter: Skill Routing for LLM Agents at Scalecited by 57
  11. 2024Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrievalcited by 50
  12. 2025ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filteringcited by 19
2 more
  1. 2025Tools are under-documented: Simple Document Expansion Boosts Tool Retrievalcited by 12
  2. 2025ScaleCall -- Agentic Tool Calling at Scale for Fintech: Challenges, Methods, and Deployment Insightscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentRouter: Heterogeneous Model Routing for Cost-Optimal Multi-Step Agentic Workflows
  2. 2026Latency-Quality Routing for Functionally Equivalent Tools in LLM Agents
  3. 2026Agent-as-a-Router: Agentic Model Routing for Coding Tasks
  4. 2026Agentic Routing: The Harness-Native Data Flywheel
  5. 2026AgentFloor: How Far Up the tool use Ladder Can Small Open-Weight Models Go?
  6. 2026How Fast Do Agents Rot? An Empirical Study of Long-Horizon Degradation in LLM Agents for Production Decision-Making
  7. 2026FlyRoute: Self-Evolving Agent Profiling via Data Flywheel for Adaptive Task Routing
  8. 2026TwinRouterBench: Fast Static and Live Dynamic Evaluation for Realistic Agentic LLM Routing