# EvoGPT: Leveraging LLM-Driven Seed Diversity to Improve Search-Based Test Suite Generation

arXiv: https://arxiv.org/abs/2505.12424 · HTML: https://mlsanity.com/paper/2505.12424/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 18 May 2025.

## Builds on (most cited first, 15 of 15)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [An Empirical Evaluation of Using Large Language Models for Automated Unit Test Generation](https://mlsanity.com/paper/2302.06527/) (2023)
- [Large Language Models as Evolutionary Optimizers](https://mlsanity.com/paper/2310.19046/) (2023)
- [ChatUniTest: A Framework for LLM-Based Test Generation](https://mlsanity.com/paper/2305.04764/) (2023)
- [Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap](https://mlsanity.com/paper/2401.10034/) (2024)
- [Pynguin: Automated Unit Test Generation for Python](https://mlsanity.com/paper/2202.05218/) (2022)
- [Insights into LLM Long-Context Failures: When Transformers Know but Don't Tell](https://mlsanity.com/paper/2406.14673/) (2024)
- [A3Test: Assertion-Augmented Automated Test Case Generation](https://mlsanity.com/paper/2302.10352/) (2023)
- [Mutation-Guided LLM-based Test Generation at Meta](https://mlsanity.com/paper/2501.12862/) (2025)
- [TestART: Improving LLM-based Unit Testing via Co-evolution of Automated Generation and Repair Iteration](https://mlsanity.com/paper/2408.03095/) (2024)
- [Test Wars: A Comparative Study of SBST, Symbolic Execution, and LLM-Based Approaches to Unit Test Generation](https://mlsanity.com/paper/2501.10200/) (2025)
- [Understanding the Role of Temperature in Diverse Question Generation by GPT-4](https://mlsanity.com/paper/2404.09366/) (2024)
- [Learning How to Search: Generating Effective Test Cases Through Adaptive Fitness Function Selection](https://mlsanity.com/paper/2102.04822/) (2021)
- [Improving the Readability of Automatically Generated Tests using Large Language Models](https://mlsanity.com/paper/2412.18843/) (2024)

## Built on by (most cited first, 2 of 2)

- [Evaluating Software Process Models for Multi-Agent Class-Level Code Generation](https://mlsanity.com/paper/2511.09794/) (2025)
- [From Pilots to Practices: A Scoping Review of GenAI-Enabled Personalization in Computer Science Education](https://mlsanity.com/paper/2512.20714/) (2025)

## Abstract (arXiv)

Search-Based Software Testing (SBST) is a well-established approach for automated unit test generation, yet it often suffers from premature convergence and limited diversity in the generated test suites. Recently, Large Language Models (LLMs) have emerged as an alternative technique for unit test generation. We present EvoGPT, a hybrid test generation system that integrates LLM-based test generation with SBST-based test suite optimization. EvoGPT uses LLMs to generate an initial population of test suites, and uses an Evolutionary Algorithm (EA) to further optimize this test suite population. A distinguishing feature of EvoGPT is its explicit enforcement of diversity, achieved through the use of multiple temperatures and prompt instructions during test generation. In addition, each LLM-generated test is refined using a generation-repair loop and coverage-guided assertion generation. To address evolutionary plateaus, EvoGPT also detects stagnation during search and injects additional LLM-generated tests aimed at previously uncovered branches. Here too diversity is enforced using multiple temperatures and prompt instructions. We evaluate EvoGPT on Defects4J, a standard benchmark for test generation. The results show that EvoGPT achieves, on average, a 10% improvement in both code coverage and mutation score metrics compared to TestART, an LLM-only baseline; and EvoSuite, a standard SBST baseline. An ablation study indicates that explicitly enforcing diversity both at initialization and during the search is key to effectively leveraging LLMs for automated unit test generation.