# What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Search

arXiv: https://arxiv.org/abs/2604.19440 · HTML: https://mlsanity.com/paper/2604.19440/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 31 of its 54 references that are in the map.
- First posted on arXiv 21 Apr 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 31)

- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery](https://mlsanity.com/paper/2408.06292/) (2024)
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization](https://mlsanity.com/paper/1911.12237/) (2019)
- [ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution](https://mlsanity.com/paper/2402.01145/) (2024)
- [Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model](https://mlsanity.com/paper/2401.02051/) (2024)
- [KernelBench: Can LLMs Write Efficient GPU Kernels?](https://mlsanity.com/paper/2502.10517/) (2025)
- [EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers](https://mlsanity.com/paper/2309.08532/) (2023)
- [A Comprehensive Survey of Self-Evolving AI Agents: A New Paradigm Bridging Foundation Models and Lifelong Agentic Systems](https://mlsanity.com/paper/2508.07407/) (2025)
- [ASSET: A Dataset for Tuning and Evaluation of Sentence Simplification Models with Multiple Rewriting Transformations](https://mlsanity.com/paper/2005.00481/) (2020)
- [Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap](https://mlsanity.com/paper/2401.10034/) (2024)
- [Symbolic Regression with a Learned Concept Library](https://mlsanity.com/paper/2409.09359/) (2024)
- [NPHardEval: Dynamic Benchmark on Reasoning Ability of Large Language Models via Complexity Classes](https://mlsanity.com/paper/2312.14890/) (2023)
- [CORAL: Towards Autonomous Multi-Agent Evolution for Open-Ended Discovery](https://mlsanity.com/paper/2604.01658/) (2026)
- [Algorithm Discovery With LLMs: Evolutionary Search Meets Reinforcement Learning](https://mlsanity.com/paper/2504.05108/) (2025)
- [Large Language Model-Based Evolutionary Optimizer: Reasoning with elitism](https://mlsanity.com/paper/2403.02054/) (2024)
- [Flows: Building Blocks of Reasoning and Collaborating AI](https://mlsanity.com/paper/2308.01285/) (2023)
- [HeurAgenix: Leveraging LLMs for Solving Complex Combinatorial Optimization Challenges](https://mlsanity.com/paper/2506.15196/) (2025)
- [LLEMA: Evolutionary Search with LLMs for Multi-Objective Materials Discovery](https://mlsanity.com/paper/2510.22503/) (2025)
- [OPT-BENCH: Evaluating LLM Agent on Large-Scale Search Spaces Optimization Problems](https://mlsanity.com/paper/2506.10764/) (2025)
- [Fitness Landscape of Large Language Model-Assisted Automated Algorithm Search](https://mlsanity.com/paper/2504.19636/) (2025)
- [T-MAP: Red-Teaming LLM Agents with Trajectory-aware Evolutionary Search](https://mlsanity.com/paper/2603.22341/) (2026)
- [A Knapsack by Any Other Name: Presentation impacts LLM performance on NP-hard problems](https://mlsanity.com/paper/2502.13776/) (2025)
- [TrajEvo: Designing Trajectory Prediction Heuristics via LLM-driven Evolution](https://mlsanity.com/paper/2505.04480/) (2025)
- [Behaviour Space Analysis of LLM-driven Meta-heuristic Discovery](https://mlsanity.com/paper/2507.03605/) (2025)
- [MolEvolve: LLM-Guided Evolutionary Search for Interpretable Molecular Optimization](https://mlsanity.com/paper/2603.24382/) (2026)
- [Pre-trained knowledge elevates large language models beyond traditional chemical reaction optimizers](https://mlsanity.com/paper/2509.00103/) (2025)
- [Large Language Model-Driven Full-Component Evolution of Adaptive Large Neighborhood Search](https://mlsanity.com/paper/2603.06996/) (2026)
- [Large Language Model-Powered Evolutionary Code Optimization on a Phylogenetic Tree](https://mlsanity.com/paper/2601.14523/) (2026)

## Built on by (most cited first, 5 of 5)

- [What Do Evolutionary Coding Agents Evolve?](https://mlsanity.com/paper/2605.20086/) (2026)
- [Prior Knowledge or Search? A Study of LLM Agents in Hardware-Aware Code Optimization](https://mlsanity.com/paper/2605.19782/) (2026)
- [Spicing up Genetic Netlist Generation with LLMs](https://mlsanity.com/paper/2608.23317/) (2026)
- [SimpleEvol: An Agent-Loop Framework for LLM-Driven Automated Heuristic Design with Minimal Human Priors](https://mlsanity.com/paper/2609.37172/) (2026)
- [Visible but Not Avoided: State-Dependent Re-Proposal of Rejected Candidates in Catalog-Based LLM Optimization Loops](https://mlsanity.com/paper/2610.15738/) (2026)

## Abstract (arXiv)

Recent work has demonstrated the promise of orchestrating large language models (LLMs) within evolutionary and agentic optimization systems. However, the mechanisms driving these optimization gains remain poorly understood. In this work, we present a large-scale study of LLM-guided evolutionary search, collecting optimization trajectories for 15 LLMs across 8 tasks. Although zero-shot problem-solving ability correlates with final optimization outcomes, it explains only part of the variance: models with similar initial capability often induce dramatically different search trajectories and outcomes. By analyzing these trajectories, we find that strong LLM optimizers behave as local refiners, producing frequent incremental improvements while progressively localizing the search in semantic space. Conversely, weaker optimizers exhibit large semantic drift, with sporadic breakthroughs followed by stagnation. Notably, various measures of solution novelty do not predict final performance; novelty is beneficial only when the search remains sufficiently localized around high-performing regions of the solution space. Our results highlight the importance of trajectory analysis for understanding and improving LLM-based optimization systems and provide actionable insights for their design and training.