mlsanityData to 7 Oct 2026

A Systematic Survey on Large Language Models for Algorithm Design

arXiv 2410.14716 ↗First posted 11 Oct 2024cs.LG cs.AI cs.CLcited by 65

Not on the map. Why?

Abstract (arXiv)

Algorithm design is crucial for effective problem-solving across various domains. The advent of Large Language Models (LLMs) has notably enhanced the automation and innovation within this field, offering new perspectives and promising solutions. In just a few years, this integration has yielded remarkable progress in areas ranging from combinatorial optimization to scientific discovery. Despite this rapid expansion, a holistic understanding of the field is hindered by the lack of a systematic review, as existing surveys either remain limited to narrow sub-fields or with different objectives. This paper seeks to provide a systematic review of algorithm design with LLMs. We introduce a taxonomy that categorises the roles of LLMs as optimizers, predictors, extractors and designers, analyzing the progress, advantages, and limitations within each category. We further synthesize literature across the three phases of the algorithm design pipeline and across diverse algorithmic applications that define the current landscape. Finally, we outline key open challenges and opportunities to guide future research. To support future research and collaboration, we provide an accompanying repository at: https://github.com/FeiLiu36/LLM4AlgorithmDesign.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 84 of its 169 references that are in the map. First posted on arXiv 11 Oct 2024.

Builds on

84 of 169 references.

References · 84

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  3. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  4. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  5. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,020
  6. 2023Large Language Modelscited by 774
  7. 2024A Survey on Large Language Models for Code Generationcited by 646
  8. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 628
  9. 2018Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizoncited by 612
  10. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 372
  11. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 348
  12. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 283
38 more
  1. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 266
  2. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 264
  3. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  4. 2022Instruction Induction: From Few Examples to Natural Language Task Descriptionscited by 152
  5. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  6. 2024Direct Preference Optimization of Video Large Multimodal Models from Language Model Rewardcited by 145
  7. 2023NL4Opt Competition: Formulating Optimization Problems Based on Their Natural Language Descriptionscited by 133
  8. 2024Large Language Models to Enhance Bayesian Optimizationcited by 127
  9. 2024ORLM: A Customizable Framework in Training Large Models for Automated Optimization Modelingcited by 124
  10. 2023Large Language Models as Evolutionary Optimizerscited by 122
  11. 2021HPOBench: A Collection of Reproducible Multi-Fidelity Benchmark Problems for HPOcited by 111
  12. 2024LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristicscited by 92
  13. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 89
  14. 2023Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planningcited by 88
  15. 2023LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimizationcited by 86
  16. 2024Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimizationcited by 84
  17. 2024LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discoverycited by 82
  18. 2015Bayesian optimization for materials designcited by 78
  19. 2023Label-free Node Classification on Graphs with Large Language Models (LLMS)cited by 72
  20. 2023Using Large Language Models for Hyperparameter Optimizationcited by 71
  21. 2024A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languagescited by 65
  22. 2023MechAgents: Large language model multi-agent collaborations can solve mechanics problems, generate new data, and integrate knowledgecited by 62
  23. 2023Auto MC-Reward: Automated Dense Reward Design with Large Language Models for Minecraftcited by 60
  24. 2023DrugAssist: A Large Language Model for Molecule Optimizationcited by 57
  25. 2024Large Language Models As Evolution Strategiescited by 57
  26. 2024HoneyComb: A Flexible LLM-Based Agent System for Materials Sciencecited by 56
  27. 2024AgentLite: A Lightweight Library for Building and Advancing Task-Oriented LLM Agent Systemcited by 53
  28. 2024ProLLaMA: A Protein Large Language Model for Multi-Task Protein Language Processingcited by 53
  29. 2024Multi-objective Evolution of Heuristic Using Large Language Modelcited by 52
  30. 2024A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules?cited by 46
  31. 2024LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generationcited by 45
  32. 2024Evolving Code with A Large Language Modelcited by 42
  33. 2024On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)cited by 40
  34. 2024Are Large Language Models Good Prompt Optimizers?cited by 37
  35. 2023Knowledge Solver: Teaching LLMs to Search for Domain Knowledge from Knowledge Graphscited by 35
  36. 2024Unleashing the Potential of Large Language Models as Prompt Optimizers: Analogical Analysis with Gradient-based Model Optimizerscited by 33
  37. 2023MeLM, a generative pretrained language modeling framework that solves forward and inverse mechanics problemscited by 29
  38. 2023LLM Performance Predictors are good initializers for Architecture Searchcited by 29

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 35; 2026: 29.

65 papers.

Sort:

By area:

  1. 2025Monte Carlo Tree Search for Comprehensive Exploration in LLM-Based Automatic Heuristic Designcited by 100
  2. 2024LLM4AD: A Platform for Algorithm Design with Large Language Modelcited by 33
  3. 2025EoH-S: Evolution of Heuristic Set using LLMs for Automated Heuristic Designcited by 32
  4. 2025ARS: Automatic Routing Solver with Large Language Modelscited by 13
  5. 2025HiFo-Prompt: Prompting with Hindsight and Foresight for LLM-based Automatic Heuristic Designcited by 13
  6. 2025CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challengescited by 11
  7. 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestrationcited by 11
  8. 2025LLaMEA-BO: A Large Language Model Evolutionary Algorithm for Automatically Generating Bayesian Optimization Algorithmscited by 10
  9. 2025BLADE: Benchmark suite for LLM-driven Automated Design and Evolution of iterative optimisation heuristicscited by 9
  10. 2025A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solvingcited by 9
  11. 2025Fitness Landscape of Large Language Model-Assisted Automated Algorithm Searchcited by 8
  12. 2025EvoCut: Strengthening Integer Programs via Evolution-Guided Language Modelscited by 8
38 more
  1. 2025Generalizable Heuristic Generation Through LLMs with Meta-Optimizationcited by 7
  2. 2026G-LNS: Generative Large Neighborhood Search for LLM-Based Automatic Heuristic Designcited by 7
  3. 2025Leveraging Large Language Models to Develop Heuristics for Emerging Optimization Problemscited by 6
  4. 2025Evolution of Optimization Algorithms for Global Placement via Large Language Modelscited by 6
  5. 2025Large Language Models and Evolutionary Computation: A Critical Review of Bidirectional Interaction, Automated Algorithm Design, and Co-Adaptive Systemscited by 6
  6. 2025Towards Universal Offline Black-Box Optimization via Learning Language Model Embeddingscited by 6
  7. 2025Fine-tuning Large Language Model for Automated Algorithm Designcited by 6
  8. 2025TrajEvo: Designing Trajectory Prediction Heuristics via LLM-driven Evolutioncited by 5
  9. 2025RedAHD: Reduction-Based End-to-End Automatic Heuristic Design with Large Language Modelscited by 4
  10. 2025Behaviour Space Analysis of LLM-driven Meta-heuristic Discoverycited by 4
  11. 2025Large Language Models in Operations Research: Methods, Applications, and Challengescited by 4
  12. 2025Experience-Guided Reflective Co-Evolution of Prompts and Heuristics for Automatic Algorithm Designcited by 4
  13. 2025BBOPlace-Bench: Benchmarking Black-Box Optimization for Chip Placementcited by 4
  14. 2026Code Evolution for Control: Synthesizing Policies via LLM-Driven Evolutionary Searchcited by 4
  15. 2025STRCMP: Integrating Graph Structural Priors with Language Models for Combinatorial Optimizationcited by 3
  16. 2025Discovering heuristics in a complex SAT solver with large language modelscited by 3
  17. 2026TIDE: Tuning-Integrated Dynamic Evolution for LLM-Based Automated Heuristic Designcited by 3
  18. 2026RankEvolve: Automating the Discovery of Retrieval Algorithms via LLM-Driven Evolutioncited by 3
  19. 2025From Understanding to Excelling: Template-Free Algorithm Design through Structural-Functional Co-Evolutioncited by 2
  20. 2026SeaEvo: Advancing Algorithm Discovery with Strategy Space Evolutioncited by 2
  21. 2026Evolving Ensemble of Agentscited by 2
  22. 2026RefineEvo: Planning-Guided Heuristic Evolution with Bidirectional Experiencecited by 2
  23. 2026VeriSimpl: Robust Optimization Modeling from Natural Language using Simplification-based Verificationcited by 2
  24. 2025EvoVLMA: Evolutionary Vision-Language Model Adaptationcited by 1
  25. 2025Evolution of Kernels: Automated RISC-V Kernel Optimization with Large Language Modelscited by 1
  26. 2026Overlooked Safety Vulnerability in LLMs: Malicious Intelligent Optimization Algorithm Request and its Jailbreakcited by 1
  27. 2026Large Language Models for Designing Participatory Budgeting Rulescited by 1
  28. 2026Rethinking Code Similarity for Automated Algorithm Design with LLMscited by 1
  29. 2026AgenticCANN: Automated Ascend C Operator Generation via Knowledge-Augmented Agentic Evolutioncited by 1
  30. 2026ATLAS: Scaffold-Free Algorithm Synthesis by LLMs via Embedding-Guided Quality-Diversity Searchcited by 1
  31. 2025SAGE: Steering Dialog Generation with Future-Aware State-Action Augmentationcited by 0
  32. 2025Large Language Models for Multi-Facility Location Mechanism Designcited by 0
  33. 2025Optimization Problem Solving Can Transition to Evolutionary Agentic Workflowscited by 0
  34. 2025LLM-Based Instance-Driven Heuristic Bias In the Context of a Biased Random Key Genetic Algorithmcited by 0
  35. 2025Learn to Relax with Large Language Models: Solving Constraint Optimization Problems via Bidirectional Coevolutioncited by 0
  36. 2025Online Operator Design in Evolutionary Optimization for Flexible Job Shop Scheduling via Large Language Modelscited by 0
  37. 2025CogMCTS: A Novel Cognitive-Guided Monte Carlo Tree Search Framework for Iterative Heuristic Evolution with Large Language Modelscited by 0
  38. 2025Behavior and Representation in Open-Weight Large Language Models for Combinatorial Optimization: From Feature Extraction to Algorithm Selectioncited by 0

50 most cited of 65 All 65 (JSON)