# FeedbackLLM: Metadata driven Multi-Agentic Language Agnostic Test Case Generator with Evolving prompt and Coverage Feedback

arXiv: https://arxiv.org/abs/2605.01264 · HTML: https://mlsanity.com/paper/2605.01264/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 31 references that are in the map.
- First posted on arXiv 2 May 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 14 of 14)

- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [Unit Test Case Generation with Transformers and Focal Context](https://mlsanity.com/paper/2009.05617/) (2020)
- [ChatUniTest: A Framework for LLM-Based Test Generation](https://mlsanity.com/paper/2305.04764/) (2023)
- [Pynguin: Automated Unit Test Generation for Python](https://mlsanity.com/paper/2202.05218/) (2022)
- [A3Test: Assertion-Augmented Automated Test Case Generation](https://mlsanity.com/paper/2302.10352/) (2023)
- [NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers](https://mlsanity.com/paper/2207.13066/) (2022)
- [Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components](https://mlsanity.com/paper/1804.06760/) (2018)
- [Reinforcement Learning from Automatic Feedback for High-Quality Unit Test Generation](https://mlsanity.com/paper/2310.02368/) (2023)
- [Requirements-driven Test Generation for Autonomous Vehicles with Machine Learning Components](https://mlsanity.com/paper/1908.01094/) (2019)
- [The Prompt Alchemist: Automated LLM-Tailored Prompt Optimization for Test Case Generation](https://mlsanity.com/paper/2501.01329/) (2025)
- [TestBench: Evaluating Class-Level Test Case Generation Capability of Large Language Models](https://mlsanity.com/paper/2409.17561/) (2024)
- [DeepREST: Automated Test Case Generation for REST APIs Exploiting Deep Reinforcement Learning](https://mlsanity.com/paper/2408.08594/) (2024)
- [Automatic High-Level Test Case Generation using Large Language Models](https://mlsanity.com/paper/2503.17998/) (2025)
- [A Tool for Test Case Scenarios Generation Using Large Language Models](https://mlsanity.com/paper/2406.07021/) (2024)

## Abstract (arXiv)

Traditional approaches to test case generation often involve manual effort and incur significant computational overhead. Additionally, these approaches are not scalable, and hence, unsuitable for complex software systems. Recently, Large Language Models (LLMs) have been applied to software testing. However, single-shot prompt engineering-based approaches tend to hallucinate and generate redundant test cases, resulting in fewer branches. To handle the above-mentioned limitations, in this paper, we propose FeedbackLLM, a novel automated language-agnostic test case generation framework based on tightly coupled two-stage approach. In the first stage, FeedbackLLM extracts the input constraints by parsing source code and generates the possible test cases. The quality of the test cases is evaluated in the second stage by the following two specialized LLM feedback agents: (i) Line Feedback Agent: extracts the metadata related to missed line executions and (ii) Branch Feedback Agent: extracts the metadata of the unexecuted branch conditions. The above agents operate in a two-stage process, communicating in tandem, and this procedure is repeated for k-steps. Further, we also introduced a redundancy prevention cache to avoid duplicate API requests and avoid unnecessary execution cycles. The performance of the proposed architecture is evaluated on the standard benchmark programs related to C and Python programs. FeedbackLLM demonstrated more line and branch coverage than baseline tools while scaling linearly in execution time.