mlsanityData to 14 Oct 2026

Algorithm Evolution Using Large Language Model

arXiv 2311.15249 ↗First posted 26 Nov 2023cs.NE cs.AI cs.LGcited by 53

Not on the map. Why?

Abstract (arXiv)

Optimization can be found in many real-life applications. Designing an effective algorithm for a specific optimization problem typically requires a tedious amount of effort from human experts with domain knowledge and algorithm design skills. In this paper, we propose a novel approach called Algorithm Evolution using Large Language Model (AEL). It utilizes a large language model (LLM) to automatically generate optimization algorithms via an evolutionary framework. AEL does algorithm-level evolution without model training. Human effort and requirements for domain knowledge can be significantly reduced. We take constructive methods for the salesman traveling problem as a test example, we show that the constructive algorithm obtained by AEL outperforms simple hand-crafted and LLM-generated heuristics. Compared with other domain deep learning model-based algorithms, these methods exhibit excellent scalability across different problem sizes. AEL is also very different from previous attempts that utilize LLMs as search operators in algorithms.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). Builds on 50 of its 88 references that are in the map. First posted on arXiv 26 Nov 2023.

Builds on

50 of 88 references.

References · 50

  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  4. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  5. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  6. 2023Teaching Large Language Models to Self-Debugcited by 988
  7. 2023Large Language Models as Optimizerscited by 864
  8. 2018Attention, Learn to Solve Routing Problems!cited by 656
  9. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  10. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 623
  11. 2022Decomposed Prompting: A Modular Approach for Solving Complex Taskscited by 558
  12. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 463
38 more
  1. 2023Augmented Language Models: a Surveycited by 403
  2. 2019AutoML: A Survey of the State-of-the-Artcited by 390
  3. 2020POMO: Policy Optimization with Multiple Optima for Reinforcement Learningcited by 348
  4. 2023Can Language Models Solve Graph Problems in Natural Language?cited by 252
  5. 2023Large Language Models as Commonsense Knowledge for Large-Scale Task Planningcited by 231
  6. 2021Learning to Optimize: A Primer and A Benchmarkcited by 208
  7. 2023ART: Automatic multi-step reasoning and tool-use for large language modelscited by 197
  8. 2019Learning Improvement Heuristics for Solving Routing Problemscited by 164
  9. 2023Neural Combinatorial Optimization with Heavy Decoder: Toward Large Scale Generalizationcited by 142
  10. 2023Chip-Chat: Challenges and Opportunities in Conversational Hardware Designcited by 140
  11. 2022Evolution through Large Modelscited by 134
  12. 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 131
  13. 2023Large Language Models as Evolutionary Optimizerscited by 124
  14. 2023Language Model Crossover: Variation through Few-Shot Promptingcited by 122
  15. 2023ChatEDA: A Large Language Model Powered Autonomous Agent for EDAcited by 122
  16. 2019Neural Large Neighborhood Search for the Capacitated Vehicle Routing Problemcited by 117
  17. 2014ParamILS: An Automatic Algorithm Configuration Frameworkcited by 89
  18. 2023LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimizationcited by 87
  19. 2023AutoML-GPT: Automatic Machine Learning with GPTcited by 81
  20. 2023Is ChatGPT the Ultimate Programming Assistant -- How far is it?cited by 76
  21. 2023Can GPT-4 Perform Neural Architecture Search?cited by 74
  22. 2021Graph Neural Network Guided Local Search for the Traveling Salesperson Problemcited by 70
  23. 2023Towards Omni-generalizable Neural Methods for Vehicle Routing Problemscited by 69
  24. 2022Pareto Set Learning for Neural Multi-objective Combinatorial Optimizationcited by 56
  25. 2022Discovering Evolution Strategies via Meta-Black-Box Optimizationcited by 44
  26. 2023H-TSP: Hierarchically Solving the Large-Scale Travelling Salesman Problemcited by 43
  27. 2023Towards Optimizing with Large Language Modelscited by 35
  28. 2023Large Language Model for Multi-objective Evolutionary Optimizationcited by 29
  29. 2023LLM Performance Predictors are good initializers for Architecture Searchcited by 29
  30. 2019Learning to Optimize in Swarmscited by 23
  31. 2021Meta-Learning-Based Deep Reinforcement Learning for Multiobjective Optimization Problemscited by 20
  32. 2023Enhancing Genetic Improvement Mutations Using Large Language Modelscited by 13
  33. 2022Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications and Open Issuescited by 12
  34. 2020Boosting Ant Colony Optimization via Solution Prediction and Machine Learningcited by 7
  35. 2022A Data-Driven Evolutionary Transfer Optimization for Expensive Problems in Dynamic Environmentscited by 6
  36. 2019Deep Learning-Aided Tabu Search Detection for Large MIMO Systemscited by 5
  37. 2023DECN: Evolution Inspired Deep Convolution Network for Black-box Optimizationcited by 3
  38. 2022Adaptive Solution Prediction for Combinatorial Optimizationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 17; 2025: 23; 2026: 13.

53 papers.

Sort:

By area:

  1. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 276
  2. 2024Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Modelcited by 268
  3. 2025Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Designcited by 109
  4. 2024LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristicscited by 97
  5. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 90
  6. 2025Large Language Models as End-to-end Combinatorial Optimization Solverscited by 28
  7. 2024Large Language Model-Based Evolutionary Optimizer: Reasoning with elitismcited by 27
  8. 2024Toward Automated Algorithm Design: A Survey and Practical Guide to Meta-Black-Box-Optimizationcited by 24
  9. 2024Neural Combinatorial Optimization Algorithms for Solving Vehicle Routing Problems: A Comprehensive Survey with Perspectivescited by 23
  10. 2024Position: Leverage Foundational Models for Black-Box Optimizationcited by 21
  11. 2024Evolve Cost-aware Acquisition Functions Using Large Language Modelscited by 17
  12. 2025HiFo-Prompt: Prompting with Hindsight and Foresight for LLM-based Automatic Heuristic Designcited by 16
38 more
  1. 2025CoCoEvo: Co-Evolution of Programs and Test Cases to Enhance Code Generationcited by 14
  2. 2024Pretrained Optimization Model for Zero-Shot Black Box Optimizationcited by 13
  3. 2024Deep Insights into Automated Optimization with Large Language Models and Evolutionary Algorithmscited by 12
  4. 2025A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solvingcited by 11
  5. 2025LLaMEA-BO: A Large Language Model Evolutionary Algorithm for Automatically Generating Bayesian Optimization Algorithmscited by 10
  6. 2025Fitness Landscape of Large Language Model-Assisted Automated Algorithm Searchcited by 9
  7. 2024Bridging Large Language Models and Optimization: A Unified Framework for Text-attributed Combinatorial Optimizationcited by 8
  8. 2025ExLLM: Experience-Enhanced LLM Optimization for Molecular Design and Beyondcited by 8
  9. 2025Multimodal LLM-assisted Evolutionary Search for Programmatic Control Policiescited by 8
  10. 2024Using Large Language Models for Parametric Shape Optimizationcited by 7
  11. 2024Can Large Language Models Solve Robot Routing?cited by 6
  12. 2025Fine-tuning Large Language Model for Automated Algorithm Designcited by 6
  13. 2025Large Language Models in Operations Research: Methods, Applications, and Challengescited by 6
  14. 2026PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMscited by 6
  15. 2026Effective Harness Engineering for Algorithm Discovery with Coding Agentscited by 6
  16. 2024tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)cited by 5
  17. 2025TrajEvo: Designing Trajectory Prediction Heuristics via LLM-driven Evolutioncited by 5
  18. 2025ACEvo: Adversarial Co-Evolution of Problem Distributions and Solvers for Combinatorial Optimizationcited by 4
  19. 2025Experience-Guided Reflective Co-Evolution of Prompts and Heuristics for Automatic Algorithm Designcited by 4
  20. 2025Can Large Language Models Be Trusted as Evolutionary Optimizers for Network-Structured Combinatorial Problems?cited by 3
  21. 2025Fully Autonomous Programming using Iterative Multi-Agent Debugging with Large Language Modelscited by 3
  22. 2025Automatic MILP Model Construction for Multi-Robot Task Allocation and Scheduling Based on Large Language Modelscited by 3
  23. 2024Adaptive Augmentation Policy Optimization with LLM Feedbackcited by 2
  24. 2025Large Language Model-Driven Surrogate-Assisted Evolutionary Algorithm for Expensive Optimizationcited by 2
  25. 2026Large Language Model-Powered Evolutionary Code Optimization on a Phylogenetic Treecited by 2
  26. 2026Game-Theoretic Co-Evolution for LLM-Based Heuristic Discoverycited by 2
  27. 2026DrugR: Optimizing Molecular Drugs through LLM-based Explicit Reasoningcited by 2
  28. 2026From Heuristic Selection to Automated Algorithm Design: LLMs Benefit from Strong Priorscited by 2
  29. 2024Evolutionary Computation for the Design and Enrichment of General-Purpose Artificial Intelligence Systems: Survey and Prospectscited by 1
  30. 2025REMoH: A Reflective Evolution of Multi-objective Heuristics approach via Large Language Modelscited by 1
  31. 2025GreenTEA: Gradient Descent with Topic-modeling and Evolutionary Auto-promptingcited by 1
  32. 2026Aligning LLMs with Graph Neural Solvers for Combinatorial Optimizationcited by 1
  33. 2026Back to the Beginning of Heuristic Design: Bridging Code and Knowledge with LLMscited by 1
  34. 2026Beyond Average Performance: Dynamic Instance Clustering and Specialized Algorithm Design in LLM-Assisted Evolutionary Searchcited by 1
  35. 2025Large Language Models for Multi-Facility Location Mechanism Designcited by 0
  36. 2025ABCDWaveNet: Advancing Robust Road Ponding Detection in Fog through Dynamic Frequency-Spatial Synergycited by 0
  37. 2025Optimization Problem Solving Can Transition to Evolutionary Agentic Workflowscited by 0
  38. 2026OR-Agent: Bridging Evolutionary Search and Structured Research for Automated Heuristic Designcited by 0

50 most cited of 53 All 53 (JSON)