# Enhancing Genetic Improvement Mutations Using Large Language Models

arXiv: https://arxiv.org/abs/2310.19813 · HTML: https://mlsanity.com/paper/2310.19813/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 17 references that are in the map.
- First posted on arXiv 18 Oct 2023.

## Builds on (most cited first, 7 of 7)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [Keep the Conversation Going: Fixing 162 out of 337 bugs for 0.42 each using ChatGPT](https://mlsanity.com/paper/2304.00385/) (2023)
- [An Analysis of the Automatic Bug Fixing Performance of ChatGPT](https://mlsanity.com/paper/2301.08653/) (2023)
- [Fuzz4All: Universal Fuzzing with Large Language Models](https://mlsanity.com/paper/2308.04748/) (2023)
- [Inductive reasoning in humans and large language models](https://mlsanity.com/paper/2306.06548/) (2023)
- [Towards Objective-Tailored Genetic Improvement Through Large Language Models](https://mlsanity.com/paper/2304.09386/) (2023)

## Built on by (most cited first, 13 of 13)

- [Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap](https://mlsanity.com/paper/2401.10034/) (2024)
- [Algorithm Evolution Using Large Language Model](https://mlsanity.com/paper/2311.15249/) (2023)
- [Search-Based LLMs for Code Optimization](https://mlsanity.com/paper/2408.12159/) (2024)
- [When Large Language Models Meet Evolutionary Algorithms: Potential Enhancements and Challenges](https://mlsanity.com/paper/2401.10510/) (2024)
- [How Multimodal Integration Boost the Performance of LLM for Optimization: Case Study on Capacitated Vehicle Routing Problems](https://mlsanity.com/paper/2403.01757/) (2024)
- [It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization](https://mlsanity.com/paper/2407.00487/) (2024)
- [Geneverse: A collection of Open-source Multimodal Large Language Models for Genomic and Proteomic Research](https://mlsanity.com/paper/2406.15534/) (2024)
- [LLM-Virus: Evolutionary Jailbreak Attack on Large Language Models](https://mlsanity.com/paper/2501.00055/) (2024)
- [Decision Tree Induction Through LLMs via Semantically-Aware Evolution](https://mlsanity.com/paper/2503.14217/) (2025)
- [Can Large Language Models Be Trusted as Evolutionary Optimizers for Network-Structured Combinatorial Problems?](https://mlsanity.com/paper/2501.15081/) (2025)
- [Structure-Aware Cooperative Ensemble Evolutionary Optimization on Combinatorial Problems with Multimodal Large Language Models](https://mlsanity.com/paper/2510.21906/) (2025)
- [The New Compiler Stack: A Survey on the Synergy of LLMs and Compilers](https://mlsanity.com/paper/2601.02045/) (2026)
- [From Hand-Crafted to LLM-Based Variation Operators in Metaheuristics: A Tutorial](https://mlsanity.com/paper/2609.31649/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have been successfully applied to software engineering tasks, including program repair. However, their application in search-based techniques such as Genetic Improvement (GI) is still largely unexplored. In this paper, we evaluate the use of LLMs as mutation operators for GI to improve the search process. We expand the Gin Java GI toolkit to call OpenAI's API to generate edits for the JCodec tool. We randomly sample the space of edits using 5 different edit types. We find that the number of patches passing unit tests is up to 75% higher with LLM-based edits than with standard Insert edits. Further, we observe that the patches found with LLMs are generally less diverse compared to standard edits. We ran GI with local search to find runtime improvements. Although many improving patches are found by LLM-enhanced GI, the best improving patch was found by standard GI.