# Leveraging Large Language Models to Improve REST API Testing

arXiv: https://arxiv.org/abs/2312.00894 · HTML: https://mlsanity.com/paper/2312.00894/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2023.

## Built on by (most cited first, 20 of 20)

- [Large Language Model (LLM) for Software Security: Code Analysis, Malware Analysis, Reverse Engineering](https://mlsanity.com/paper/2504.07137/) (2025)
- [A Framework for Testing and Adapting REST APIs as LLM Tools](https://mlsanity.com/paper/2504.15546/) (2025)
- [Turbulence: Systematically and Automatically Testing Instruction-Tuned Large Language Models for Code](https://mlsanity.com/paper/2312.14856/) (2023)
- [A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputs](https://mlsanity.com/paper/2411.07098/) (2024)
- [Automating a Complete Software Test Process Using LLMs: An Automotive Case Study](https://mlsanity.com/paper/2502.04008/) (2025)
- [DAWN: Designing Distributed Agents in a Worldwide Network](https://mlsanity.com/paper/2410.22339/) (2024)
- [LogiAgent: Automated Logical Testing for REST Systems with LLM-Based Multi-Agents](https://mlsanity.com/paper/2503.15079/) (2025)
- [CodeAssistBench (CAB): Dataset & Benchmarking for Multi-turn Chat-Based Code Assistance](https://mlsanity.com/paper/2507.10646/) (2025)
- [LlamaRestTest: Effective REST API Testing with Small Language Models](https://mlsanity.com/paper/2501.08598/) (2025)
- [Advanced System Integration: Analyzing OpenAPI Chunking for Retrieval-Augmented Generation](https://mlsanity.com/paper/2411.19804/) (2024)
- [AutoRestTest: A Tool for Automated REST API Testing Using LLMs and MARL](https://mlsanity.com/paper/2501.08600/) (2025)
- [Combining TSL and LLM to Automate REST API Testing: A Comparative Study](https://mlsanity.com/paper/2509.05540/) (2025)
- [Generative transformations and patterns in LLM-native approaches for software verification and falsification](https://mlsanity.com/paper/2404.09384/) (2024)
- [Retrieval-Augmented Generation for Service Discovery: Chunking Strategies and Benchmarking](https://mlsanity.com/paper/2505.19310/) (2025)
- [Large Language Models for Software Testing: A Research Roadmap](https://mlsanity.com/paper/2509.25043/) (2025)
- [Selection of Prompt Engineering Techniques for Code Generation through Predicting Code Complexity](https://mlsanity.com/paper/2409.16416/) (2024)
- [Search-Based Software Engineering and AI Foundation Models: Current Landscape and Future Roadmap](https://mlsanity.com/paper/2505.19625/) (2025)
- [MASTEST: A LLM-Based Multi-Agent System For RESTful API Tests](https://mlsanity.com/paper/2511.18038/) (2025)
- [LLM-Powered Workflow Optimization for Multidisciplinary Software Development: An Automotive Industry Case Study](https://mlsanity.com/paper/2603.21439/) (2026)
- [LLM-Based Robustness Testing of Microservice Applications: An Empirical Study](https://mlsanity.com/paper/2605.14202/) (2026)

## Abstract (arXiv)

The widespread adoption of REST APIs, coupled with their growing complexity and size, has led to the need for automated REST API testing tools. Current tools focus on the structured data in REST API specifications but often neglect valuable insights available in unstructured natural-language descriptions in the specifications, which leads to suboptimal test coverage. Recently, to address this gap, researchers have developed techniques that extract rules from these human-readable descriptions and query knowledge bases to derive meaningful input values. However, these techniques are limited in the types of rules they can extract and prone to produce inaccurate results. This paper presents RESTGPT, an innovative approach that leverages the power and intrinsic context-awareness of Large Language Models (LLMs) to improve REST API testing. RESTGPT takes as input an API specification, extracts machine-interpretable rules, and generates example parameter values from natural-language descriptions in the specification. It then augments the original specification with these rules and values. Our evaluations indicate that RESTGPT outperforms existing techniques in both rule extraction and value generation. Given these promising results, we outline future research directions for advancing REST API testing through LLMs.