mlsanityData to 14 Oct 2026

Enrich-Retrieve-Rank: Scaling Capability Discovery Beyond In-Context Routing

arXiv 2608.22695 ↗First posted 24 Aug 2026cs.CL cs.AI cs.IRcited by 0

Not on the map. Why?

Abstract (arXiv)

Agent ecosystems now include thousands of MATS components (Models, Agents, Tools, and Skills), yet their discovery still relies on in-context routing. These systems read a registry (names, hints, or descriptions, as context budget permits), pick a candidate, invoke it, and retry on failure. This pattern degrades with scale, and registries are growing fast. We recast capability discovery as search over a registry by defining an offline enrichment step that turns sparse metadata into searchable profiles, and an online retrieve-then-rank pipeline that returns a ranked shortlist without invoking any candidates online. We show that from N=10 to 7,278 capabilities, in-context routing's top-1 accuracy (Match@1) collapses (0.85 to 0.12), while retrieve-then-rank degrades more gently (0.81 to 0.39) because its reranker still ranks the right capability first 0.70-0.87 of the time once retrieval finds it. In the Nova Micro sweep, the crossover is around N=500. We compare against two in-context baselines. Full-Ctx puts the whole registry in the prompt and asks the LLM to pick. Search&Pick gives the LLM a search tool to narrow candidates before it picks. At full scale the pipeline leads Search&Pick by 6.5 percentage points (pp) on Match@1 at about half the cost. It reduces cost 70x versus Full-Ctx. We use a fixed configuration (same enrichment, retriever, and scorer weights) across agent, tool, and skill registries. The pipeline runs in production as the default capability-discovery layer of a large-scale multi-agent platform.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 33 references that are in the map. First posted on arXiv 24 Aug 2026.

Builds on

16 of 33 references.

References · 16

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  2. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  3. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  4. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  5. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
  6. 2023C-Pack: Packed Resources For General Chinese Embeddingscited by 578
  7. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 517
  8. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
  9. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 240
  10. 2019Document Expansion by Query Predictioncited by 231
  11. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Usecited by 217
  12. 2024EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instructioncited by 154
4 more
  1. 2023CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsetscited by 129
  2. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 124
  3. 2024Planning and Editing What You Retrieve for Enhanced Tool Learningcited by 21
  4. 2025Tools are under-documented: Simple Document Expansion Boosts Tool Retrievalcited by 12

Built on by

Not cited yet.