mlsanityData to 14 Oct 2026

Schema First Tool APIs for LLM Agents: A Controlled Study of Tool Misuse, Recovery, and Budgeted Performance

arXiv 2603.13404 ↗First posted 12 Mar 2026cs.SE cs.AIcited by 7

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Tool use has become central to modern LLM agents, yet interface design is rarely isolated as an experimental variable. This paper studies whether schema based tool contracts and structured validation diagnostics improve reliability under strict interaction budgets. We evaluate three conditions that preserve identical tool semantics and information content: free form documentation, JSON Schema specifications, and JSON Schema with structured diagnostics. We implement a deterministic software engineering sandbox with logs, metrics, configurations, and repository tasks, and evaluate a fully crossed pilot with one open local model, three seeds, three interface conditions, and four budgets. We report end task success, interface misuse, execution failures, semantic misuse, recovery behavior, and overhead. In this pilot, success remains zero across conditions, while schema conditions reduce interface misuse but not semantic misuse. The evidence supports a precise interpretation that interface formalization improves contract adherence, but semantic action quality and timeout sensitive tasks remain dominant bottlenecks under constrained local inference.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 12 of its 15 references that are in the map. First posted on arXiv 12 Mar 2026.

Builds on

12 of 15 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Gorilla: Large Language Model Connected with Massive APIs24 May
  2. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs14 Apr
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Tools9 Feb

Other references · 9

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  6. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  7. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  8. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
  9. 2023Augmented Language Models: a Surveycited by 403

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026ContractBench: Can LLM Agents Preserve Observation Contracts?
  2. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration
  3. 2026TSCG: Deterministic Tool-Schema Compilation for Agentic LLM Deployments
  4. 2026When JSON Is Not Enough: Semantic Reliability of Schema-Constrained LLM Ordering Agents
  5. 2026Harness Engineering in LLM Tool Use via Agent-Native Reusable Tool Primitives
  6. 2026How Consistent Are LLM Agents? Measuring Behavioral Reproducibility in Multi-Step Tool-Calling Pipelines
  7. 2026Action-Space Shaping for LLM Agents: Measuring and Mitigating Tool-Schema Bias
  8. 2026MANTRA: Synthesizing SMT-Validated Compliance Benchmarks for Tool-Using LLM Agents