# KAT: Dependency-aware Automated API Testing with Large Language Models

arXiv: https://arxiv.org/abs/2407.10227 · HTML: https://mlsanity.com/paper/2407.10227/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jul 2024.

## Built on by (most cited first, 6 of 6)

- [A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputs](https://mlsanity.com/paper/2411.07098/) (2024)
- [Automating a Complete Software Test Process Using LLMs: An Automotive Case Study](https://mlsanity.com/paper/2502.04008/) (2025)
- [Evaluating LLMs on Sequential API Call Through Automated Test Generation](https://mlsanity.com/paper/2507.09481/) (2025)
- [Disrupting Test Development with AI Assistants](https://mlsanity.com/paper/2411.02328/) (2024)
- [MASTEST: A LLM-Based Multi-Agent System For RESTful API Tests](https://mlsanity.com/paper/2511.18038/) (2025)
- [Mining Workflow Graphs for Black-Box Boundary Testing of Conversational LLM Agents](https://mlsanity.com/paper/2607.06873/) (2026)

## Abstract (arXiv)

API testing has increasing demands for software companies. Prior API testing tools were aware of certain types of dependencies that needed to be concise between operations and parameters. However, their approaches, which are mostly done manually or using heuristic-based algorithms, have limitations due to the complexity of these dependencies. In this paper, we present KAT (Katalon API Testing), a novel AI-driven approach that leverages the large language model GPT in conjunction with advanced prompting techniques to autonomously generate test cases to validate RESTful APIs. Our comprehensive strategy encompasses various processes to construct an operation dependency graph from an OpenAPI specification and to generate test scripts, constraint validation scripts, test cases, and test data. Our evaluation of KAT using 12 real-world RESTful services shows that it can improve test coverage, detect more undocumented status codes, and reduce false positives in these services in comparison with a state-of-the-art automated test generation tool. These results indicate the effectiveness of using the large language model for generating test scripts and data for API testing.