# Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap

arXiv: https://arxiv.org/abs/2401.10034 · HTML: https://mlsanity.com/paper/2401.10034/

## Key facts

- Cited by 90 papers in the map (data through 14 Oct 2026).
- Builds on 93 of its 199 references that are in the map.
- First posted on arXiv 18 Jan 2024.

## Builds on (most cited first, 30 of 93)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [ChatGPT Outperforms Crowd-Workers for Text-Annotation Tasks](https://mlsanity.com/paper/2303.15056/) (2023)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models](https://mlsanity.com/paper/2310.04451/) (2023)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [WizardCoder: Empowering Code Large Language Models with Evol-Instruct](https://mlsanity.com/paper/2306.08568/) (2023)
- [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://mlsanity.com/paper/2209.07858/) (2022)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [Eureka: Human-Level Reward Design via Coding Large Language Models](https://mlsanity.com/paper/2310.12931/) (2023)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [InCoder: A Generative Model for Code Infilling and Synthesis](https://mlsanity.com/paper/2204.05999/) (2022)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [Illuminating search spaces by mapping elites](https://mlsanity.com/paper/1504.04909/) (2015)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)

## Built on by (most cited first, 30 of 90)

- [ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution](https://mlsanity.com/paper/2402.01145/) (2024)
- [LLM-SR: Scientific Equation Discovery via Programming with Large Language Models](https://mlsanity.com/paper/2404.18400/) (2024)
- [Open-Endedness is Essential for Artificial Superhuman Intelligence](https://mlsanity.com/paper/2406.04268/) (2024)
- [A Systematic Survey on Large Language Models for Algorithm Design](https://mlsanity.com/paper/2410.14716/) (2024)
- [Meta Context Engineering via Agentic Skill Evolution](https://mlsanity.com/paper/2601.21557/) (2026)
- [Diversity-Aware Policy Optimization for Large Language Model Reasoning](https://mlsanity.com/paper/2505.23433/) (2025)
- [LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizers](https://mlsanity.com/paper/2503.14434/) (2025)
- [Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Models](https://mlsanity.com/paper/2412.02980/) (2024)
- [Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learning](https://mlsanity.com/paper/2509.24372/) (2025)
- [Evolutionary Large Language Model for Automated Feature Transformation](https://mlsanity.com/paper/2405.16203/) (2024)
- [Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence](https://mlsanity.com/paper/2410.11163/) (2024)
- [Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimization](https://mlsanity.com/paper/2411.00625/) (2024)
- [How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training](https://mlsanity.com/paper/2502.11196/) (2025)
- [Understanding the Importance of Evolutionary Search in Automated Heuristic Design with Large Language Models](https://mlsanity.com/paper/2407.10873/) (2024)
- [Design Principle Transfer in Neural Architecture Search via Large Language Models](https://mlsanity.com/paper/2408.11330/) (2024)
- [Nature-Inspired Population-Based Evolution of Large Language Models](https://mlsanity.com/paper/2503.01155/) (2025)
- [Deep Insights into Automated Optimization with Large Language Models and Evolutionary Algorithms](https://mlsanity.com/paper/2410.20848/) (2024)
- [A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solving](https://mlsanity.com/paper/2509.08269/) (2025)
- [It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization](https://mlsanity.com/paper/2407.00487/) (2024)
- [Fitness Landscape of Large Language Model-Assisted Automated Algorithm Search](https://mlsanity.com/paper/2504.19636/) (2025)
- [L-AutoDA: Leveraging Large Language Models for Automated Decision-based Adversarial Attacks](https://mlsanity.com/paper/2401.15335/) (2024)
- [From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Models](https://mlsanity.com/paper/2407.09502/) (2024)
- [HM3: Hierarchical Multi-Objective Model Merging for Pretrained Models](https://mlsanity.com/paper/2409.18893/) (2024)
- [ExLLM: Experience-Enhanced LLM Optimization for Molecular Design and Beyond](https://mlsanity.com/paper/2502.12845/) (2025)
- [LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discovery](https://mlsanity.com/paper/2506.00844/) (2025)
- [Using Large Language Models for Parametric Shape Optimization](https://mlsanity.com/paper/2412.08072/) (2024)
- [Position: Scaling LLM Agents Requires Asymptotic Analysis with LLM Primitives](https://mlsanity.com/paper/2502.04358/) (2025)
- [iDSE: Navigating Design Space Exploration in High-Level Synthesis Using LLMs](https://mlsanity.com/paper/2505.22086/) (2025)
- [SPARQ: Synthetic Problem Generation for Reasoning via Quality-Diversity Algorithms](https://mlsanity.com/paper/2506.06499/) (2025)
- [Large Language Models and Evolutionary Computation: A Critical Review of Bidirectional Interaction, Automated Algorithm Design, and Co-Adaptive Systems](https://mlsanity.com/paper/2505.15741/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have not only revolutionized natural language processing but also extended their prowess to various domains, marking a significant stride towards artificial general intelligence. The interplay between LLMs and evolutionary algorithms (EAs), despite differing in objectives and methodologies, share a common pursuit of applicability in complex problems. Meanwhile, EA can provide an optimization framework for LLM's further enhancement under black-box settings, empowering LLM with flexible global search capacities. On the other hand, the abundant domain knowledge inherent in LLMs could enable EA to conduct more intelligent searches. Furthermore, the text processing and generative capabilities of LLMs would aid in deploying EAs across a wide range of tasks. Based on these complementary advantages, this paper provides a thorough review and a forward-looking roadmap, categorizing the reciprocal inspiration into two main avenues: LLM-enhanced EA and EA-enhanced LLM. Some integrated synergy methods are further introduced to exemplify the complementarity between LLMs and EAs in diverse scenarios, including code generation, software engineering, neural architecture search, and various generation tasks. As the first comprehensive review focused on the EA research in the era of LLMs, this paper provides a foundational stepping stone for understanding the collaborative potential of LLMs and EAs. The identified challenges and future directions offer guidance for researchers and practitioners to unlock the full potential of this innovative collaboration in propelling advancements in optimization and artificial intelligence. We have created a GitHub repository to index the relevant papers: https://github.com/wuxingyu-ai/LLM4EC.