mlsanityData to 14 Oct 2026

MASTEST: A LLM-Based Multi-Agent System For RESTful API Tests

arXiv 2511.18038 ↗First posted 22 Nov 2025cs.SE cs.AI cs.LGcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Testing RESTful API is increasingly complicated but indispensable to quality assurance of cloud-native applications. This paper reports a multi-agent system called MASTEST that combines LLM-based intelligent agents and programmed agents to automate REST API testing. They form a complete tool chain covering the whole workflow of REST API test with API specification in the OpenAPI Swagger format as the input. It also incorporates human testers in the process to review and correct LLM generated test artefacts to control the quality of testing activities. MASTEST is evaluated on two LLMs, GPT-4o and DeepSeek V3.1 Reasoner with five public APIs. Its performances on various testing activities are measured by a wide range of metrics, including adequacy and coverage metrics, the syntax and data type correctness of generated test scripts, the usability of LLM generated test cases and scripts, as well as the bug detection ability. Experiment results demonstrated that both DeepSeek and GPT-4o achieved a high overall performance but had strengths and weaknesses on different testing activities. MASTEST generated test cases achieved 94% and 98% unit test coverage and 79% and 78% system test coverage for GPT-4o and DeepSeek respectively in comparison with human designed test cases. The generated test scripts maintained 100% syntax correctness and only required minimal manual edits for semantic correctness. The generated test scripts contain assertions on the expected status code as well as contents in the response messages. They are highly capable of detecting bugs in the REST APIs. Experiment data shows that the bug detection rates are between 2.13 to 4.50 per operation. These findings indicate that MASTEST is highly efficient and effective.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 15 of its 32 references that are in the map. First posted on arXiv 22 Nov 2025.

Builds on

15 of 32 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025LogiAgent: Automated Logical Testing for REST Systems with LLM-Based Multi-Agents19 Mar
  2. 2024A Multi-Agent Approach for REST API Testing with Semantic Graphs and LLM-Driven Inputs11 Nov

Other references · 13

  1. 2023Leveraging Large Language Models to Improve REST API Testingcited by 20
  2. 2022Morest: Model-based RESTful API Testing with Execution Feedbackcited by 11
  3. 2022Testing RESTful APIs: A Surveycited by 11
  4. 2024DeepREST: Automated Test Case Generation for REST APIs Exploiting Deep Reinforcement Learningcited by 8
  5. 2024APITestGenie: Automated API Test Generation through Generative AIcited by 7
  6. 2024KAT: Dependency-aware Automated API Testing with Large Language Modelscited by 6
  7. 2019QuickREST: Property-based Test Generation of OpenAPI-Described RESTful APIscited by 4
  8. 2024Automating REST API Postman Test Cases Using LLMcited by 4
  9. 2025LlamaRestTest: Effective REST API Testing with Small Language Modelscited by 4
  10. 2025AutoRestTest: A Tool for Automated REST API Testing Using LLMs and MARLcited by 3
  11. 2025Combining TSL and LLM to Automate REST API Testing: A Comparative Studycited by 3
  12. 2019RESTful API Automated Test Case Generationcited by 2
1 more
  1. 2024FuzzTheREST: An Intelligent Automated Black-box RESTful API Fuzzercited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Test Amplification for REST APIs via Single and Multi-Agent LLM Systems
  2. 2025A Framework for Testing and Adapting REST APIs as LLM Tools
  3. 2025Agentic LLMs for REST API Test Amplification: A Comparative Study Across Cloud Applications
  4. 2026Multi-Agent LLM-based Metamorphic Testing for REST APIs
  5. 2026Multi-Agent LLM Collaboration for Unit Test Generation via Human-Testing-Inspired Workflows
  6. 2025Challenges in Testing Large Language Model Based Software: A Faceted Taxonomy
  7. 2026Tangent: An Empirical Study of Testing Practices for LLM-Based Agent Applications
  8. 2024The Potential of LLMs in Automating Software Testing: From Generation to Reporting