# ToolFactory: Automating Tool Generation by Leveraging LLM to Understand REST API Documentations

arXiv: https://arxiv.org/abs/2501.16945 · HTML: https://mlsanity.com/paper/2501.16945/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jan 2025.

## Built on by (most cited first, 9 of 9)

- [LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions](https://mlsanity.com/paper/2509.18970/) (2025)
- [From REST to MCP: An Empirical Study of API Wrapping and Automated Server Generation for LLM Agents](https://mlsanity.com/paper/2507.16044/) (2025)
- [Advanced Tool Learning and Selection System (ATLASS): A Closed-Loop Framework Using LLM](https://mlsanity.com/paper/2503.10071/) (2025)
- [Security Considerations for Multi-agent Systems](https://mlsanity.com/paper/2603.09002/) (2026)
- [NaviAgent: Graph-Driven Bilevel Planning for Scalable Tool Orchestration](https://mlsanity.com/paper/2506.19500/) (2025)
- [Skill2Query: Exploiting Skill Structure to Generate Pseudo-Queries for Agent Skill Retrieval](https://mlsanity.com/paper/2608.16071/) (2026)
- [HarnessAPI: A Skill-First Framework for Unified Streaming APIs and MCP Tools](https://mlsanity.com/paper/2605.22733/) (2026)
- [Improving Proficiency and Efficiency of Android GUI Agents via Self-Generating Tool Actions](https://mlsanity.com/paper/2609.06792/) (2026)
- [DADL: A Declarative Description Language for Enterprise Tool Libraries in LLM Agent Systems](https://mlsanity.com/paper/2605.05247/) (2026)

## Abstract (arXiv)

LLM-based tool agents offer natural language interfaces, enabling users to seamlessly interact with computing services. While REST APIs are valuable resources for building such agents, they must first be transformed into AI-compatible tools. Automatically generating AI-compatible tools from REST API documents can greatly streamline tool agent development and minimize user learning curves. However, API documentation often suffers from a lack of standardization, inconsistent schemas, and incomplete information. To address these issues, we developed ToolFactory, an open-source pipeline for automating tool generation from unstructured API documents. To enhance the reliability of the developed tools, we implemented an evaluation method to diagnose errors. Furthermore, we built a knowledge base of verified tools, which we leveraged to infer missing information from poorly documented APIs. We developed the API Extraction Benchmark, comprising 167 API documents and 744 endpoints in various formats, and designed a JSON schema to annotate them. This annotated dataset was utilized to train and validate ToolFactory. The experimental results highlight the effectiveness of ToolFactory. We also demonstrated ToolFactory by creating a domain-specific AI agent for glycomaterials research. ToolFactory exhibits significant potential for facilitating the seamless integration of scientific REST APIs into AI workflows.