# SpeCrawler: Generating OpenAPI Specifications from API Documentation Using Large Language Models

arXiv: https://arxiv.org/abs/2402.11625 · HTML: https://mlsanity.com/paper/2402.11625/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 16 references that are in the map.
- First posted on arXiv 18 Feb 2024.

## Builds on (most cited first, 3 of 3)

- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Automated Test Generation for REST APIs: No Time to Rest Yet](https://mlsanity.com/paper/2204.08348/) (2022)
- [Automatically Extracting Web API Specifications from HTML Documentation](https://mlsanity.com/paper/1801.08928/) (2018)

## Built on by (most cited first, 2 of 2)

- [From Specification to Service: Accelerating API-First Development Using Multi-Agent Systems](https://mlsanity.com/paper/2510.19274/) (2025)
- [Agent4Spec: An LLM-based Agentic System to Optimize API Specifications through Automated Validation and Market Analysis](https://mlsanity.com/paper/2610.13273/) (2026)

## Abstract (arXiv)

In the digital era, the widespread use of APIs is evident. However, scalable utilization of APIs poses a challenge due to structure divergence observed in online API documentation. This underscores the need for automatic tools to facilitate API consumption. A viable approach involves the conversion of documentation into an API Specification format. While previous attempts have been made using rule-based methods, these approaches encountered difficulties in generalizing across diverse documentation. In this paper we introduce SpeCrawler, a comprehensive system that utilizes large language models (LLMs) to generate OpenAPI Specifications from diverse API documentation through a carefully crafted pipeline. By creating a standardized format for numerous APIs, SpeCrawler aids in streamlining integration processes within API orchestrating systems and facilitating the incorporation of tools into LLMs. The paper explores SpeCrawler's methodology, supported by empirical evidence and case studies, demonstrating its efficacy through LLM capabilities.