mlsanityData to 7 Oct 2026

Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap

arXiv 2401.10034 ↗First posted 18 Jan 2024cs.NE cs.AI cs.CLcited by 89

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have not only revolutionized natural language processing but also extended their prowess to various domains, marking a significant stride towards artificial general intelligence. The interplay between LLMs and evolutionary algorithms (EAs), despite differing in objectives and methodologies, share a common pursuit of applicability in complex problems. Meanwhile, EA can provide an optimization framework for LLM's further enhancement under black-box settings, empowering LLM with flexible global search capacities. On the other hand, the abundant domain knowledge inherent in LLMs could enable EA to conduct more intelligent searches. Furthermore, the text processing and generative capabilities of LLMs would aid in deploying EAs across a wide range of tasks. Based on these complementary advantages, this paper provides a thorough review and a forward-looking roadmap, categorizing the reciprocal inspiration into two main avenues: LLM-enhanced EA and EA-enhanced LLM. Some integrated synergy methods are further introduced to exemplify the complementarity between LLMs and EAs in diverse scenarios, including code generation, software engineering, neural architecture search, and various generation tasks. As the first comprehensive review focused on the EA research in the era of LLMs, this paper provides a foundational stepping stone for understanding the collaborative potential of LLMs and EAs. The identified challenges and future directions offer guidance for researchers and practitioners to unlock the full potential of this innovative collaboration in propelling advancements in optimization and artificial intelligence. We have created a GitHub repository to index the relevant papers: https://github.com/wuxingyu-ai/LLM4EC.

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). Builds on 93 of its 199 references that are in the map. First posted on arXiv 18 Jan 2024.

Builds on

93 of 199 references.

References · 93

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2022Emergent Abilities of Large Language Modelscited by 2,483
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  10. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  11. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  12. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
38 more
  1. 2019Attention is not Explanationcited by 916
  2. 2023A Survey on Multimodal Large Language Modelscited by 898
  3. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  4. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  5. 2023Large Language Models as Optimizerscited by 830
  6. 2023Large Language Modelscited by 774
  7. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 738
  8. 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
  9. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  10. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  11. 2023Unifying Large Language Models and Knowledge Graphs: A Roadmapcited by 663
  12. 2023Baseline Defenses for Adversarial Attacks Against Aligned Language Modelscited by 608
  13. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  14. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  15. 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
  16. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  17. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  18. 2015Illuminating search spaces by mapping elitescited by 406
  19. 2023A Survey on Large Language Models for Recommendationcited by 378
  20. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  21. 2023Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attackscited by 268
  22. 2024Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Modelcited by 258
  23. 2022Black-Box Tuning for Language-Model-as-a-Servicecited by 254
  24. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
  25. 2014Robots that can adapt like animalscited by 203
  26. 2023Multimodal Large Language Models: A Surveycited by 193
  27. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 189
  28. 2019The Evolved Transformercited by 180
  29. 2023Large Language Models for Software Engineering: Survey and Open Problemscited by 178
  30. 2020A Survey on Evolutionary Neural Architecture Searchcited by 142
  31. 2022Evolution through Large Modelscited by 131
  32. 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 129
  33. 2023Open Sesame! Universal Black Box Jailbreaking of Large Language Modelscited by 128
  34. 2022GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Modelscited by 127
  35. 2023Large Language Models as Evolutionary Optimizerscited by 122
  36. 2023Language Model Crossover: Variation through Few-Shot Promptingcited by 119
  37. 2022Large Language Models are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Modelscited by 102
  38. 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 89

50 most cited of 93 All 93 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 22; 2025: 39; 2026: 28.

89 papers.

Sort:

By area:

  1. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 266
  2. 2024LLM-SR: Scientific Equation Discovery via Programming with Large Language Modelscited by 117
  3. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 79
  4. 2024A Systematic Survey on Large Language Models for Algorithm Designcited by 65
  5. 2026Meta Context Engineering via Agentic Skill Evolutioncited by 42
  6. 2025Diversity-Aware Policy Optimization for Large Language Model Reasoningcited by 39
  7. 2025LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizerscited by 35
  8. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Modelscited by 31
  9. 2024Evolutionary Large Language Model for Automated Feature Transformationcited by 29
  10. 2025Evolution Strategies at Scale: LLM Fine-Tuning Beyond Reinforcement Learningcited by 28
  11. 2024Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligencecited by 27
  12. 2024Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimizationcited by 23
38 more
  1. 2024Understanding the Importance of Evolutionary Search in Automated Heuristic Design with Large Language Modelscited by 22
  2. 2025How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Trainingcited by 22
  3. 2024Design Principle Transfer in Neural Architecture Search via Large Language Modelscited by 14
  4. 2025Nature-Inspired Population-Based Evolution of Large Language Modelscited by 14
  5. 2024Deep Insights into Automated Optimization with Large Language Models and Evolutionary Algorithmscited by 12
  6. 2024It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimizationcited by 10
  7. 2025A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solvingcited by 9
  8. 2024L-AutoDA: Leveraging Large Language Models for Automated Decision-based Adversarial Attackscited by 8
  9. 2024From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Modelscited by 8
  10. 2024HM3: Hierarchical Multi-Objective Model Merging for Pretrained Modelscited by 8
  11. 2025Fitness Landscape of Large Language Model-Assisted Automated Algorithm Searchcited by 8
  12. 2024Using Large Language Models for Parametric Shape Optimizationcited by 7
  13. 2025Position: Scaling LLM Agents Requires Asymptotic Analysis with LLM Primitivescited by 7
  14. 2025LLM Cannot Discover Causality, and Should Be Restricted to Non-Decisional Support in Causal Discoverycited by 7
  15. 2025SPARQ: Synthetic Problem Generation for Reasoning via Quality-Diversity Algorithmscited by 7
  16. 2025ExLLM: Experience-Enhanced LLM Optimization for Molecular Design and Beyondcited by 6
  17. 2025Large Language Models and Evolutionary Computation: A Critical Review of Bidirectional Interaction, Automated Algorithm Design, and Co-Adaptive Systemscited by 6
  18. 2025iDSE: Navigating Design Space Exploration in High-Level Synthesis Using LLMscited by 6
  19. 2025CoLLM-NAS: Collaborative Large Language Models for Efficient Knowledge-Guided Neural Architecture Searchcited by 6
  20. 2024Multi-View Subgraph Neural Networks: Self-Supervised Learning with Scarce Labeled Datacited by 5
  21. 2025LLM-Guided Evolution: An Autonomous Model Optimization for Object Detectioncited by 5
  22. 2025TrajEvo: Designing Trajectory Prediction Heuristics via LLM-driven Evolutioncited by 5
  23. 2025TrajEvo: Trajectory Prediction Heuristics Design via LLM-driven Evolutioncited by 5
  24. 2026PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMscited by 5
  25. 2024Identify Critical Nodes in Complex Network with Large Language Modelscited by 4
  26. 2024A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Modelcited by 4
  27. 2025Pareto-Grid-Guided Large Language Models for Fast and High-Quality Heuristics Design in Multi-Objective Combinatorial Optimizationcited by 4
  28. 2025Large Language Models in Operations Research: Methods, Applications, and Challengescited by 4
  29. 2025AutoEP: LLMs-Driven Automation of Hyperparameter Evolution for Metaheuristic Algorithmscited by 4
  30. 2026What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Searchcited by 4
  31. 2026Mutation Without Variation: Convergence Dynamics in LLM-Driven Program Evolutioncited by 4
  32. 2025Can Large Language Models Be Trusted as Evolutionary Optimizers for Network-Structured Combinatorial Problems?cited by 3
  33. 2025When Large Language Models Meet Law: Dual-Lens Taxonomy, Technical Advances, and Ethical Governancecited by 3
  34. 2026TIDE: Tuning-Integrated Dynamic Evolution for LLM-Based Automated Heuristic Designcited by 3
  35. 2025EvoGPT: Leveraging LLM-Driven Seed Diversity to Improve Search-Based Test Suite Generationcited by 2
  36. 2025PhaseNAS: Language-Model Driven Architecture Search with Dynamic Phase Adaptationcited by 2
  37. 2025FELA: A Multi-Agent Evolutionary System for Feature Engineering of Industrial Event Log Datacited by 2
  38. 2026ContextEvolve: Multi-Agent Context Compression for Systems Code Optimizationcited by 2

50 most cited of 89 All 89 (JSON)