# Mutation Without Variation: Convergence Dynamics in LLM-Driven Program Evolution

arXiv: https://arxiv.org/abs/2606.05408 · HTML: https://mlsanity.com/paper/2606.05408/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 55 references that are in the map.
- First posted on arXiv 3 Jun 2026.

## Builds on (most cited first, 22 of 22)

- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [The Curse of Recursion: Training on Generated Data Makes Models Forget](https://mlsanity.com/paper/2305.17493/) (2023)
- [ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution](https://mlsanity.com/paper/2402.01145/) (2024)
- [ShinkaEvolve: Towards Open-Ended And Sample-Efficient Program Evolution](https://mlsanity.com/paper/2509.19349/) (2025)
- [Language Model Crossover: Variation through Few-Shot Prompting](https://mlsanity.com/paper/2302.12170/) (2023)
- [LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics](https://mlsanity.com/paper/2405.20132/) (2024)
- [Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap](https://mlsanity.com/paper/2401.10034/) (2024)
- [Large Language Models As Evolution Strategies](https://mlsanity.com/paper/2402.18381/) (2024)
- [Evolving Code with A Large Language Model](https://mlsanity.com/paper/2401.07102/) (2024)
- [Discovering Preference Optimization Algorithms with and for Large Language Models](https://mlsanity.com/paper/2406.08414/) (2024)
- [A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts](https://mlsanity.com/paper/2311.08374/) (2023)
- [Unveiling Attractor Cycles in Large Language Models: A Dynamical Systems View of Successive Paraphrasing](https://mlsanity.com/paper/2502.15208/) (2025)
- [Cultural evolution in populations of Large Language Models](https://mlsanity.com/paper/2403.08882/) (2024)
- [LLM as a Broken Telephone: Iterative Generation Distorts Information](https://mlsanity.com/paper/2502.20258/) (2025)
- [Code Evolution Graphs: Understanding Large Language Model Driven Design of Algorithms](https://mlsanity.com/paper/2503.16668/) (2025)
- [Digital Red Queen: Adversarial Program Evolution in Core War with LLMs](https://mlsanity.com/paper/2601.03335/) (2026)
- [An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms](https://mlsanity.com/paper/2408.02451/) (2024)
- [The Telephone Game: Evaluating Semantic Drift in Unified Models](https://mlsanity.com/paper/2509.04438/) (2025)
- [Code Evolution for Control: Synthesizing Policies via LLM-Driven Evolutionary Search](https://mlsanity.com/paper/2601.06845/) (2026)

## Built on by (most cited first, 4 of 4)

- [If It's Not Buggy, Don't Fix It: On the Dynamics of Iterative Bug-fixing with LLMs](https://mlsanity.com/paper/2609.10123/) (2026)
- [Operator Packages, Proposer Strength, and Construction-Family Plateaus in Office-Scale Verified Search](https://mlsanity.com/paper/2609.29636/) (2026)
- [Evolving Towards Better Codes: LLM-Guided Search for High-Distance Binary Linear Codes](https://mlsanity.com/paper/2609.37056/) (2026)
- [Towards Better Exploration in Sequential Test-Time Scaling](https://mlsanity.com/paper/2609.39632/) (2026)

## Abstract (arXiv)

When an LLM repeatedly mutates a program, does it explore new forms or circle back to the same ones? We study this question by analyzing LLM-driven mutation chains in the absence of selection pressure within a domain-specific language, varying prompt design, model family, and stochastic replication. We find that LLM-based mutation consistently converges toward restricted attractor regions in program space. Convergence is especially severe at the structural level: in 87% of chains, over 93% of mutations revisit a previously seen structural form, with most variation confined to terminal substitutions within recurring templates. Cycle analysis reveals short cycles and self-loops dominating the transition structure. The rate of convergence varies with prompt wording and model choice, but the phenomenon is robust across conditions. A classical GP subtree mutation operator does not exhibit comparable convergence, suggesting that the effect is intrinsic to the LLM mutation pipeline. These findings reveal a tension at the heart of LLM-driven program evolution: the same capabilities that enable semantics-aware program transformation also carry a systematic bias toward structural homogeneity that must be accounted for if such systems are to sustain open-ended exploration. Source code is available at https://github.com/can-gurkan/lmca.