mlsanityData to 14 Oct 2026

LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimization

arXiv 2306.01102 ↗First posted 1 Jun 2023cs.NE cs.AI cs.CLcited by 87

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have emerged as powerful tools capable of accomplishing a broad spectrum of tasks. Their abilities span numerous areas, and one area where they have made a significant impact is in the domain of code generation. Here, we propose using the coding abilities of LLMs to introduce meaningful variations to code defining neural networks. Meanwhile, Quality-Diversity (QD) algorithms are known to discover diverse and robust solutions. By merging the code-generating abilities of LLMs with the diversity and robustness of QD solutions, we introduce textttLLMatic, a Neural Architecture Search (NAS) algorithm. While LLMs struggle to conduct NAS directly through prompts, textttLLMatic uses a procedural approach, leveraging QD for prompts and network architecture to create diverse and high-performing networks. We test textttLLMatic on the CIFAR-10 and NAS-bench-201 benchmarks, demonstrating that it can produce competitive networks while evaluating just 2,000 candidates, even without prior knowledge of the benchmark domain or exposure to any previous top-performing models for the benchmark. The open-sourced code is available in urlhttps://github.com/umair-nasir14/LLMatic.

Key facts. Cited by 87 papers in the map (data through 14 Oct 2026). Builds on 32 of its 55 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

32 of 55 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  7. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  9. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  10. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  11. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  12. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
20 more
  1. 2015Illuminating search spaces by mapping elitescited by 425
  2. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  3. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
  4. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 344
  5. 2014Robots that can adapt like animalscited by 204
  6. 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 195
  7. 2020A Survey on Evolutionary Neural Architecture Searchcited by 142
  8. 2023Neural Architecture Search: Insights from 1000 Paperscited by 139
  9. 2022Evolution through Large Modelscited by 134
  10. 2023EvoPrompting: Language Models for Code-Level Neural Architecture Searchcited by 131
  11. 2017Quality and Diversity Optimization: A Unifying Modular Frameworkcited by 113
  12. 2022The CLRS Algorithmic Reasoning Benchmarkcited by 106
  13. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 98
  14. 2023Level Generation Through Large Language Modelscited by 68
  15. 2016Using Centroidal Voronoi Tessellations to Scale Up the Multi-dimensional Archive of Phenotypic Elites Algorithmcited by 66
  16. 2022NAS-Bench-Suite: NAS Evaluation is (Now) Surprisingly Easycited by 36
  17. 2023Practical PCG Through Large Language Modelscited by 21
  18. 2022Lambda-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cellscited by 15
  19. 2022Geographical Distance Is The New Hyperparameter: A Case Study Of Finding The Optimal Pre-trained Language For English-isiZulu Machine Translationcited by 5
  20. 2022Augmentative Topology Agents For Open-Ended Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 25; 2025: 29; 2026: 27.

87 papers.

Sort:

By area:

  1. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  2. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
  3. 2024Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Modelcited by 268
  4. 2024AgentSquare: Automatic LLM Agent Search in Modular Design Spacecited by 155
  5. 2025MLGym: A New Framework and Benchmark for Advancing AI Research Agentscited by 124
  6. 2023When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunitiescited by 90
  7. 2024Evolutionary Computation in the Era of Large Language Model: Survey and Roadmapcited by 90
  8. 2024A Systematic Survey on Large Language Models for Algorithm Designcited by 69
  9. 2024Multi-objective Evolution of Heuristic Using Large Language Modelcited by 54
  10. 2023Algorithm Evolution Using Large Language Modelcited by 53
  11. 2024EDT: Improving Large Language Models' Generation by Entropy-based Dynamic Temperature Samplingcited by 45
  12. 2024Evolving Code with A Large Language Modelcited by 43
38 more
  1. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 35
  2. 2024Surveying the Effects of Quality, Diversity, and Complexity in Synthetic Data From Large Language Modelscited by 35
  3. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  4. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  5. 2024REvolve: Reward Evolution with Large Language Models using Human Feedbackcited by 23
  6. 2024FunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearchcited by 23
  7. 2024Position: Leverage Foundational Models for Black-Box Optimizationcited by 21
  8. 2024When Large Language Models Meet Evolutionary Algorithms: Potential Enhancements and Challengescited by 20
  9. 2024Word2World: Generating Stories and Worlds through Large Language Modelscited by 18
  10. 2024Design Principle Transfer in Neural Architecture Search via Large Language Modelscited by 17
  11. 2025Neural Architecture Discovery via Autonomous Evolutioncited by 16
  12. 2024Adaptive Learn-then-Test: Statistically Valid and Efficient Hyperparameter Selectioncited by 15
  13. 2024Pretrained Optimization Model for Zero-Shot Black Box Optimizationcited by 13
  14. 2025A Systematic Survey on Large Language Models for Evolutionary Optimization: From Modeling to Solvingcited by 11
  15. 2024Generalists vs. Specialists: Evaluating LLMs on Highly-Constrained Biophysical Sequence Optimization Taskscited by 10
  16. 2025Enhancing LLM-Based Neural Network Generation: Few-Shot Prompting and Efficient Validation for Automated Architecture Designcited by 10
  17. 2026From Memorization to Creativity: LLM as a Designer of Novel Neural Architecturescited by 10
  18. 2024From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Modelscited by 8
  19. 2024GameTraversalBenchmark: Evaluating Planning Abilities Of Large Language Models Through Traversing 2D Game Mapscited by 8
  20. 2024NADER: Neural Architecture Design via Multi-Agent Collaborationcited by 8
  21. 2024Grammar-based Game Description Generation using Large Language Modelscited by 7
  22. 2025SEKI: Self-Evolution and Knowledge Inspiration based Neural Architecture Search via Large Language Modelscited by 7
  23. 2025iDSE: Navigating Design Space Exploration in High-Level Synthesis Using LLMscited by 7
  24. 2025LM-Searcher: Cross-domain Neural Architecture Search with LLMs via Unified Numerical Encodingcited by 7
  25. 2023ACES: Generating Diverse Programming Puzzles with with Autotelic Generative Modelscited by 6
  26. 2025Decision Tree Induction Through LLMs via Semantically-Aware Evolutioncited by 6
  27. 2025Large Language Models and Evolutionary Computation: A Critical Review of Bidirectional Interaction, Automated Algorithm Design, and Co-Adaptive Systemscited by 6
  28. 2025CoLLM-NAS: Collaborative Large Language Models for Efficient Knowledge-Guided Neural Architecture Searchcited by 6
  29. 2026Closed-Loop LLM Discovery of Non-Standard Channel Priors in Vision Modelscited by 6
  30. 2026PathWise: Planning through World Model for Automated Heuristic Design via Self-Evolving LLMscited by 6
  31. 2024QUBE: Enhancing Automatic Heuristic Design via Quality-Uncertainty Balanced Evolutioncited by 5
  32. 2025LLM-Guided Evolution: An Autonomous Model Optimization for Object Detectioncited by 5
  33. 2025Can LLMs Revolutionize the Design of Explainable and Efficient TinyML Models?cited by 5
  34. 2025Transferrable Surrogates in Expressive Neural Architecture Search Spacescited by 5
  35. 2024OStr-DARTS: Differentiable Neural Architecture Search based on Operation Strengthcited by 4
  36. 2025LLM-Guided Search for Deletion-Correcting Codescited by 4
  37. 2026Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffscited by 3
  38. 2025MONAQ: Multi-Objective Neural Architecture Querying for Time-Series Analysis on Resource-Constrained Devicescited by 2

50 most cited of 87 All 87 (JSON)