mlsanityData to 7 Oct 2026

Evolving Code with A Large Language Model

arXiv 2401.07102 ↗First posted 13 Jan 2024cs.NE cs.AIcited by 42

Not on the map. Why?

Abstract (arXiv)

Algorithms that use Large Language Models (LLMs) to evolve code arrived on the Genetic Programming (GP) scene very recently. We present LLM GP, a formalized LLM-based evolutionary algorithm designed to evolve code. Like GP, it uses evolutionary operators, but its designs and implementations of those operators radically differ from GP's because they enlist an LLM, using prompting and the LLM's pre-trained pattern matching and sequence completion capability. We also present a demonstration-level variant of LLM GP and share its code. By addressing algorithms that range from the formal to hands-on, we cover design and LLM-usage considerations as well as the scientific challenges that arise when using an LLM for genetic programming.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 36 of its 52 references that are in the map. First posted on arXiv 13 Jan 2024.

Builds on

36 of 52 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  10. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,739
  11. 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 940
  12. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
24 more
  1. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  2. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 521
  3. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
  4. 2021Do Prompt-Based Models Really Understand the Meaning of their Prompts?cited by 324
  5. 2023How is ChatGPT's behavior changing over time?cited by 265
  6. 2021Sustainable AI: Environmental Implications, Challenges and Opportunitiescited by 242
  7. 2022Evolution through Large Modelscited by 131
  8. 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 129
  9. 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 128
  10. 2023Language Model Crossover: Variation through Few-Shot Promptingcited by 119
  11. 2022Teaching Algorithmic Reasoning via In-context Learningcited by 111
  12. 2023An Analysis of the Automatic Bug Fixing Performance of ChatGPTcited by 108
  13. 2023The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domaincited by 97
  14. 2020Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditingcited by 89
  15. 2023LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimizationcited by 86
  16. 2022Formal Algorithms for Transformerscited by 70
  17. 2023MarioGPT: Open-Ended Text2Level Generation through Large Language Modelscited by 63
  18. 2023Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Modelscited by 61
  19. 2023Fully Autonomous Programming with Large Language Modelscited by 31
  20. 2021Choose Your Programming Copilot: A Comparison of the Program Synthesis Performance of GitHub Copilot and Genetic Programmingcited by 24
  21. 2023Fluid Transformers and Creative Analogies: Exploring Large Language Models' Capacity for Augmenting Cross-Domain Analogical Creativitycited by 23
  22. 2023ChatGPT and Other Large Language Models as Evolutionary Engines for Online Interactive Collaborative Game Designcited by 20
  23. 2023Evaluating statistical language models as pragmatic reasonerscited by 14
  24. 2023Evolutionary Verbalizer Search for Prompt-based Few Shot Text Classificationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 11; 2025: 18; 2026: 13.

42 papers.

Sort:

By area:

  1. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 763
  2. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 266
  3. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 89
  4. 2024A Systematic Survey on Large Language Models for Algorithm Designcited by 65
  5. 2026EvoX: Meta-Evolution for Automated Discoverycited by 45
  6. 2024LLM4AD: A Platform for Algorithm Design with Large Language Modelcited by 33
  7. 2024Understanding the Importance of Evolutionary Search in Automated Heuristic Design with Large Language Modelscited by 22
  8. 2024Evolve Cost-aware Acquisition Functions Using Large Language Modelscited by 17
  9. 2025Large Language Models (LLMs) for Source Code Analysis: applications, models and datasetscited by 15
  10. 2024Deep Insights into Automated Optimization with Large Language Models and Evolutionary Algorithmscited by 12
  11. 2025Cognitive Alpha Mining via LLM-Driven Code-Based Evolutioncited by 12
  12. 2024It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimizationcited by 10
30 more
  1. 2025DISC: Dynamic Decomposition Improves LLM Inference Scalingcited by 9
  2. 2025Fitness Landscape of Large Language Model-Assisted Automated Algorithm Searchcited by 8
  3. 2025Multimodal LLM-assisted Evolutionary Search for Programmatic Control Policiescited by 8
  4. 2024In Context Learning and Reasoning for Symbolic Regression with Large Language Modelscited by 7
  5. 2025(X)-evolve: Solution space evolution powered by large language modelscited by 6
  6. 2024tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)cited by 5
  7. 2026PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMscited by 5
  8. 2026Evolutionary System Prompt Learning for Reinforcement Learning in LLMscited by 5
  9. 2025Improving Existing Optimization Algorithms with LLMscited by 4
  10. 2025RedAHD: Reduction-Based End-to-End Automatic Heuristic Design with Large Language Modelscited by 4
  11. 2025Evolving Excellence: Automated Optimization of LLM-based Agentscited by 4
  12. 2026Mutation Without Variation: Convergence Dynamics in LLM-Driven Program Evolutioncited by 4
  13. 2026The AI Telco Engineer: Toward Autonomous Discovery of Wireless Communications Algorithmscited by 3
  14. 2026MEMENTO: Memory-Guided Memetic Code-as-Policy Evolutioncited by 3
  15. 2025Industrial LLM-based Code Optimization under Regulation: A Mixture-of-Agents Approachcited by 2
  16. 2026Game-Theoretic Co-Evolution for LLM-Based Heuristic Discoverycited by 2
  17. 2026Structured Progressive Knowledge Activation for LLM-Driven Neural Architecture Searchcited by 2
  18. 2024Evolutionary Computation for the Design and Enrichment of General-Purpose Artificial Intelligence Systems: Survey and Prospectscited by 1
  19. 2025REMoH: A Reflective Evolution of Multi-objective Heuristics approach via Large Language Modelscited by 1
  20. 2025EvoSpeak: Large Language Models for Interpretable Genetic Programming-Evolved Heuristicscited by 1
  21. 2026Evolutionary Discovery of Reinforcement Learning Algorithms via Large Language Modelscited by 1
  22. 2026See it to Place it: Evolving Macro Placements with Vision-Language Modelscited by 1
  23. 2025Instructing the Architecture Search for Spatial-temporal Sequence Forecasting with LLMcited by 0
  24. 2025MEMETRON: Metaheuristic Mechanisms for Test-time Response Optimization of Large Language Modelscited by 0
  25. 2025An In-depth Study of LLM Contributions to the Bin Packing Problemcited by 0
  26. 2025AutoICE: Automatically Synthesizing Verifiable C Code via LLM-driven Evolutioncited by 0
  27. 2025CogMCTS: A Novel Cognitive-Guided Monte Carlo Tree Search Framework for Iterative Heuristic Evolution with Large Language Modelscited by 0
  28. 2026DyACE: Dynamic Algorithm Co-evolution for Online Automated Heuristic Design with Large Language Modelcited by 0
  29. 2026CDEoH: Category-Driven Automatic Algorithm Design With Large Language Modelscited by 0
  30. 2026Budget-Aware LLM Discovery via Cost-Calibrated Frontier Utilitycited by 0