mlsanityData to 7 Oct 2026

Tool-Schema Compression Enables Agentic RAG Under Constrained Context Budgets

arXiv 2605.26165 ↗First posted 24 May 2026cs.SE cs.AI cs.CLcited by 0

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

also in 1 more
Abstract (arXiv)

Agentic RAG systems that equip language models with dozens to hundreds of tool definitions face a critical resource conflict: tool schemas consume the same context window needed for retrieval-augmented generation. We present the first systematic study of this tool-context trade-off, evaluating 14 models spanning 1.5B-32B local models plus one frontier API model across 6,566 controlled API calls at three context budgets (8K, 16K, 32K) with 28 tool definitions. Applying TSCG conservative-profile compression (44-50% schema token savings), we observe a binary enablement effect: at 8K tokens, JSON-schema tool definitions overflow the context window entirely, yielding near-zero EM (2.6% average), while compressed schemas restore RAG functionality with +20.5 pp average exact-match lift across all eight models (+24.7 pp among the six exhibiting full enablement). At 32K -- where both formats fit -- four of five tested models show delta <= 1 pp, confirming the effect is purely budget-driven. External validation on HotpotQA (50 multi-hop questions) shows +48 pp EM under the same overflow scenario. Frontier scaling tests demonstrate that JSON schemas overflow at ~494 tools while compressed schemas remain operational beyond 800 tools. Our results establish tool-schema compression as a necessary infrastructure layer for agentic RAG in constrained-context deployments. All code, data, and checkpoints are publicly available.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 11 of its 19 references that are in the map. First posted on arXiv 24 May 2026.

Builds on

11 of 19 references.

References · 11

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  6. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  8. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
  9. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  10. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  11. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Tool Attention Is All You Need: Dynamic Tool Gating and Lazy Schema Loading for Eliminating the MCP/Tools Tax in Scalable Agentic Workflows
  2. 2026TSCG: Deterministic Tool-Schema Compilation for Agentic LLM Deployments
  3. 2026SchemaRouter: Field-Aware Tool Routing for Efficient Heterogeneous Agentic RAG
  4. 2026A-RAG: Scaling Agentic Retrieval-Augmented Generation via Hierarchical Retrieval Interfaces
  5. 2026ToolSpec: Accelerating Tool Calling via Schema-Aware and Retrieval-Augmented Speculative Decoding
  6. 2026Mapping the RAG Landscape: A Four Axis Taxonomy of Efficiency, Defense, Interactivity, and Reasoning
  7. 2026Is Agentic RAG worth it? An experimental comparison of RAG approaches
  8. 2026Structured Context Engineering for File-Native Agentic Systems: Evaluating Schema Accuracy, Format Effectiveness, and Multi-File Navigation at Scale