mlsanityData to 14 Oct 2026

EvoPrompting: Language Models for Code-Level Neural Architecture Search

arXiv 2302.14838 ↗First posted 28 Feb 2023cs.NE cs.AI cs.CLcited by 131

Not on the map. Why?

Cited by 27 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Given the recent impressive accomplishments of language models (LMs) for code generation, we explore the use of LMs as adaptive mutation and crossover operators for an evolutionary neural architecture search (NAS) algorithm. While NAS still proves too difficult a task for LMs to succeed at solely through prompting, we find that the combination of evolutionary prompt engineering with soft prompt-tuning, a method we term EvoPrompting, consistently finds diverse and high performing models. We first demonstrate that EvoPrompting is effective on the computationally efficient MNIST-1D dataset, where EvoPrompting produces convolutional architecture variants that outperform both those designed by human experts and naive few-shot prompting in terms of accuracy and model size. We then apply our method to searching for graph neural networks on the CLRS Algorithmic Reasoning Benchmark, where EvoPrompting is able to design novel architectures that outperform current state-of-the-art models on 21 out of 30 algorithmic reasoning tasks while maintaining similar model size. EvoPrompting is successful at designing accurate and efficient neural network architectures across a variety of machine learning tasks, while also being general enough for easy adaptation to other tasks beyond neural network design.

Key facts. Cited by 131 papers in the map (data through 14 Oct 2026). Builds on 51 of its 74 references that are in the map. First posted on arXiv 28 Feb 2023.

Builds on

51 of 74 references.

References · 51

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  6. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  7. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  8. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  9. 2021Program Synthesis with Large Language Modelscited by 3,637
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  11. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  12. 2020Extracting Training Data from Large Language Modelscited by 2,189
38 more
  1. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  3. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  4. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  5. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  6. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,392
  7. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  8. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  9. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,195
  10. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  11. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  12. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  13. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  14. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  15. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  16. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  17. 2021Unified Pre-training for Program Understanding and Generationcited by 366
  18. 2022A Systematic Evaluation of Large Language Models of Codecited by 363
  19. 2017Large-Scale Evolution of Image Classifierscited by 348
  20. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 254
  21. 2019The Evolved Transformercited by 180
  22. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 164
  23. 2019What Can Neural Networks Reason About?cited by 159
  24. 2018Efficient Multi-objective Neural Architecture Search via Lamarckian Evolutioncited by 153
  25. 2022GitHub Copilot AI pair programmer: Asset or Liability?cited by 152
  26. 2020AutoML-Zero: Evolving Machine Learning Algorithms From Scratchcited by 143
  27. 2022Evolution through Large Modelscited by 134
  28. 2023Language Model Crossover: Variation through Few-Shot Promptingcited by 122
  29. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
  30. 2022Language Model Cascadescited by 107
  31. 2022The CLRS Algorithmic Reasoning Benchmarkcited by 106
  32. 2020NATS-Bench: Benchmarking NAS Algorithms for Architecture Topology and Sizecited by 98
  33. 2022Towards Learning Universal Hyperparameter Optimizers with Transformerscited by 81
  34. 2022Limitations of Language Models in Arithmetic and Symbolic Inductioncited by 71
  35. 2020Can weight sharing outperform random architecture search? An investigation with TuNAScited by 68
  36. 2020Evolving Normalization-Activation Layerscited by 45
  37. 2021Generating Fake Cyber Threat Intelligence Using Transformer-Based Modelscited by 29
  38. 2022Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Modelscited by 21

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 16; 2024: 40; 2025: 35; 2026: 40.

131 papers.

Sort:

By area:

  1. 2025AlphaEvolve: A coding agent for scientific and algorithmic discoverycited by 867
  2. 2023Large Language Models as Optimizerscited by 864
  3. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  4. 2023Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolutioncited by 551
  5. 2024ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolutioncited by 276
  6. 2024Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Modelcited by 268
  7. 2024Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Promptscited by 205
  8. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 193
  9. 2024RE-Bench: Evaluating frontier AI R&D capabilities of language model agents against human expertscited by 175
  10. 2024AgentSquare: Automatic LLM Agent Search in Modular Design Spacecited by 155
  11. 2023Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generationcited by 148
  12. 2024Large Language Models to Enhance Bayesian Optimizationcited by 135
38 more
  1. 2024LLM-SR: Scientific Equation Discovery via Programming with Large Language Modelscited by 133
  2. 2024EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithmscited by 113
  3. 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 90
  4. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 90
  5. 2023LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimizationcited by 87
  6. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 80
  7. 2023Can GPT-4 Perform Neural Architecture Search?cited by 74
  8. 2023Using Large Language Models for Hyperparameter Optimizationcited by 71
  9. 2024Large Language Model Agent for Hyper-Parameter Optimizationcited by 64
  10. 2023Algorithm Evolution Using Large Language Modelcited by 53
  11. 2024Efficient Prompting Methods for Large Language Models: A Surveycited by 47
  12. 2024LLaMoCo: Instruction Tuning of Large Language Models for Optimization Code Generationcited by 45
  13. 2024Evolving Code with A Large Language Modelcited by 43
  14. 2024Discovering Preference Optimization Algorithms with and for Large Language Modelscited by 42
  15. 2023Quality-Diversity through AI Feedbackcited by 40
  16. 2025LLM-FE: Automated Feature Engineering for Tabular Data with LLMs as Evolutionary Optimizerscited by 38
  17. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 35
  18. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Modelscited by 35
  19. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  20. 2023Large Language Model for Multi-objective Evolutionary Optimizationcited by 29
  21. 2024LICO: Large Language Models for In-Context Molecular Optimizationcited by 26
  22. 2026AVO: Agentic Variation Operators for Autonomous Evolutionary Searchcited by 24
  23. 2024FunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearchcited by 23
  24. 2024Understanding the Importance of Evolutionary Search in Automated Heuristic Design with Large Language Modelscited by 22
  25. 2024Position: Leverage Foundational Models for Black-Box Optimizationcited by 21
  26. 2025A Survey on LLM Test-Time Compute via Search: Tasks, LLM Profiling, Search Algorithms, and Relevant Frameworkscited by 21
  27. 2024When Large Language Models Meet Evolutionary Algorithms: Potential Enhancements and Challengescited by 20
  28. 2025Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMscited by 17
  29. 2023Why Can Large Language Models Generate Correct Chain-of-Thoughts?cited by 16
  30. 2025AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Taskscited by 16
  31. 2026JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness Evolutioncited by 16
  32. 2025CoCoEvo: Co-Evolution of Programs and Test Cases to Enhance Code Generationcited by 14
  33. 2024Pretrained Optimization Model for Zero-Shot Black Box Optimizationcited by 13
  34. 2025NNGPT: Rethinking AutoML with Large Language Modelscited by 13
  35. 2024Deep Insights into Automated Optimization with Large Language Models and Evolutionary Algorithmscited by 12
  36. 2025A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solvingcited by 11
  37. 2026M^star: Every Task Deserves Its Own Memory Harnesscited by 11
  38. 2024Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Taskscited by 10

50 most cited of 131 All 131 (JSON)