# A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving

arXiv: https://arxiv.org/abs/2509.08269 · HTML: https://mlsanity.com/paper/2509.08269/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 175 of its 268 references that are in the map.
- First posted on arXiv 10 Sep 2025.

## Builds on (most cited first, 30 of 175)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [BloombergGPT: A Large Language Model for Finance](https://mlsanity.com/paper/2303.17564/) (2023)
- [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey](https://mlsanity.com/paper/2403.14608/) (2024)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Deep Reinforcement Learning: An Overview](https://mlsanity.com/paper/1701.07274/) (2017)
- [ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution](https://mlsanity.com/paper/2402.01145/) (2024)
- [Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model](https://mlsanity.com/paper/2401.02051/) (2024)
- [Can Language Models Solve Graph Problems in Natural Language?](https://mlsanity.com/paper/2305.10037/) (2023)
- [Reinforcement Learning for Combinatorial Optimization: A Survey](https://mlsanity.com/paper/2003.03600/) (2020)
- [Large Language Models in Finance: A Survey](https://mlsanity.com/paper/2311.10723/) (2023)

## Built on by (most cited first, 11 of 11)

- [Large Language Models in Operations Research: Methods, Applications, and Challenges](https://mlsanity.com/paper/2509.18180/) (2025)
- [Generalizing Test-time Compute-optimal Scaling as an Optimizable Graph](https://mlsanity.com/paper/2511.00086/) (2025)
- [Detect and Act: Automated Dynamic Optimizer through Meta-Black-Box Optimization](https://mlsanity.com/paper/2601.22542/) (2026)
- [ATLAS: Scaffold-Free Algorithm Synthesis by LLMs via Embedding-Guided Quality-Diversity Search](https://mlsanity.com/paper/2608.15546/) (2026)
- [PortAgent: LLM-driven Vehicle Dispatching Agent for Port Terminals](https://mlsanity.com/paper/2512.14417/) (2025)
- [AgentLTV: An Agent-Based Unified Search-and-Evolution Framework for Automated Lifetime Value Prediction](https://mlsanity.com/paper/2602.21634/) (2026)
- [Evolving Parallel Algorithm Portfolios via Potential-Aware Instance Generation with LLMs](https://mlsanity.com/paper/2608.06808/) (2026)
- [SOVER: Formal Certification of Optimization Reformulations via LLM-Assisted SMT Verification](https://mlsanity.com/paper/2609.00728/) (2026)
- [Evolutionary Safety of Recursive Self-Improving AI: Taxonomy, Risk Discovery, and Evaluation](https://mlsanity.com/paper/2609.31186/) (2026)
- [From Hand-Crafted to LLM-Based Variation Operators in Metaheuristics: A Tutorial](https://mlsanity.com/paper/2609.31649/) (2026)
- [OptiCom : A Unified Framework for State-Conditioned Composition in LLM-Driven Optimization](https://mlsanity.com/paper/2609.37221/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly integrated with evolutionary computation to support optimization tasks. This survey primarily focuses on evolutionary optimization, i.e., optimization based on evolutionary computation. For brevity, we use the term optimization throughout to denote this scope. However, existing surveys typically examine isolated roles of LLMs and do not provide a unified view that connects optimization modeling with optimization solving. To address this gap, we systematically review recent developments through a workflow-oriented framework. First, we organize the literature into two primary stages: LLMs for optimization modeling and LLMs for optimization solving (in this survey, the terms optimization modeling and optimization solving are used as concise forms of optimization problem modeling and optimization problem solving, respectively). Second, we divide the solving stage into three paradigms according to the role of the LLM: stand-alone optimizers, low-level components embedded in optimization algorithms, and high-level managers for algorithm selection and generation. Third, we analyze representative methods, identify their technical limitations, and clarify their relationships with traditional optimization approaches. We further substantiate this taxonomy through benchmark systematization, baseline comparisons, and practitioner-oriented guidance, and we review interdisciplinary applications across the natural sciences, engineering, and machine learning. Based on the resulting analysis, we identify research directions toward dynamic, self-evolving, and agentic optimization ecosystems. An up-to-date collection of related literature is maintained at https://github.com/ishmael233/LLM4OPT.