# PPO guided Agentic Pipeline for Adaptive Prompt Selection and Test Case Generation

arXiv: https://arxiv.org/abs/2605.00942 · HTML: https://mlsanity.com/paper/2605.00942/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 1 May 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 14 of 14)

- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [Unit Test Case Generation with Transformers and Focal Context](https://mlsanity.com/paper/2009.05617/) (2020)
- [ChatUniTest: A Framework for LLM-Based Test Generation](https://mlsanity.com/paper/2305.04764/) (2023)
- [Pynguin: Automated Unit Test Generation for Python](https://mlsanity.com/paper/2202.05218/) (2022)
- [A3Test: Assertion-Augmented Automated Test Case Generation](https://mlsanity.com/paper/2302.10352/) (2023)
- [NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers](https://mlsanity.com/paper/2207.13066/) (2022)
- [Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components](https://mlsanity.com/paper/1804.06760/) (2018)
- [Reinforcement Learning from Automatic Feedback for High-Quality Unit Test Generation](https://mlsanity.com/paper/2310.02368/) (2023)
- [The Prompt Alchemist: Automated LLM-Tailored Prompt Optimization for Test Case Generation](https://mlsanity.com/paper/2501.01329/) (2025)
- [TestBench: Evaluating Class-Level Test Case Generation Capability of Large Language Models](https://mlsanity.com/paper/2409.17561/) (2024)
- [DeepREST: Automated Test Case Generation for REST APIs Exploiting Deep Reinforcement Learning](https://mlsanity.com/paper/2408.08594/) (2024)
- [Automated Control Logic Test Case Generation using Large Language Models](https://mlsanity.com/paper/2405.01874/) (2024)
- [Automatic High-Level Test Case Generation using Large Language Models](https://mlsanity.com/paper/2503.17998/) (2025)
- [A Tool for Test Case Scenarios Generation Using Large Language Models](https://mlsanity.com/paper/2406.07021/) (2024)

## Abstract (arXiv)

Developing effective test cases capable of thoroughly exercising large-scale software systems is inherently difficult, especially if such systems have voluminous, complex, and deeply nested source codes. In this work, we present a novel approach for generating test cases using a reinforcement learning-driven agentic framework where Proximal Policy Optimization (PPO) is coupled with an LLM engine to guide prompt selection during test generation. Our approach consists of two phases. In Phase I, the ToT-guided optimization agent partitions and minimizes the source code by removing redundancies without changing the functional behavior of the source code. In Phase II, a PPO-based policy network is trained to solve the problem of selecting prompts among eight different prompting techniques, such as Boundary Value Analysis, Random Fuzzing, etc., based on the inputted 11-dimensional state vector representing the source code complexity metrics and live coverage metrics to direct the LLM engine towards exploring unvisited paths in the program. The PPO agent receives rewards based on a combination of increases in line and branch coverages, penalties for unexplored branches, and rewards for reducing source code length. From experiments conducted on twenty benchmark programs, it is evident that the proposed approach, PPO-LLM, outperforms CBMC, kS-LLM, and kS-LLM++ in terms of branch and line coverage in almost all cases, for various loop bound values ranging from BOUND~1 to BOUND~2000. While at BOUND~1, the coverage of branches is 100% using PPO-LLM on the PALS suite, in comparison, it is around 86.8% using kS-LLM++. This confirms that adaptive prompt selection driven by PPO substantially outperforms static prompting strategies on PALS type programs.