# Design Principle Transfer in Neural Architecture Search via Large Language Models

arXiv: https://arxiv.org/abs/2408.11330 · HTML: https://mlsanity.com/paper/2408.11330/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 21 Aug 2024.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search](https://mlsanity.com/paper/2001.00326/) (2020)
- [SNAS: Stochastic Neural Architecture Search](https://mlsanity.com/paper/1812.09926/) (2018)
- [Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search](https://mlsanity.com/paper/1911.12126/) (2019)
- [Evolutionary Computation in the Era of Large Language Model: Survey and Roadmap](https://mlsanity.com/paper/2401.10034/) (2024)
- [LLMatic: Neural Architecture Search via Large Language Models and Quality Diversity Optimization](https://mlsanity.com/paper/2306.01102/) (2023)
- [AutoML-GPT: Automatic Machine Learning with GPT](https://mlsanity.com/paper/2305.02499/) (2023)
- [Meta-Learning of Neural Architectures for Few-Shot Learning](https://mlsanity.com/paper/1911.11090/) (2019)
- [Neural Architecture Transfer](https://mlsanity.com/paper/2005.05859/) (2020)
- [TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search](https://mlsanity.com/paper/2105.11871/) (2021)
- [Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets](https://mlsanity.com/paper/2107.00860/) (2021)
- [LLM Performance Predictors are good initializers for Architecture Search](https://mlsanity.com/paper/2310.16712/) (2023)
- [Stronger NAS with Weaker Predictors](https://mlsanity.com/paper/2102.10490/) (2021)
- [Large Language Model-driven Meta-structure Discovery in Heterogeneous Information Network](https://mlsanity.com/paper/2402.11518/) (2024)
- [Accelerating Evolutionary Neural Architecture Search via Multi-Fidelity Evaluation](https://mlsanity.com/paper/2108.04541/) (2021)
- [Heterogeneous Graph Neural Architecture Search with GPT-4](https://mlsanity.com/paper/2312.08680/) (2023)
- [Visual Analysis of Neural Architecture Spaces for Summarizing Design Principles](https://mlsanity.com/paper/2208.09665/) (2022)

## Built on by (most cited first, 17 of 17)

- [SEKI: Self-Evolution and Knowledge Inspiration based Neural Architecture Search via Large Language Models](https://mlsanity.com/paper/2502.20422/) (2025)
- [iDSE: Navigating Design Space Exploration in High-Level Synthesis Using LLMs](https://mlsanity.com/paper/2505.22086/) (2025)
- [Transferrable Surrogates in Expressive Neural Architecture Search Spaces](https://mlsanity.com/paper/2504.12971/) (2025)
- [Structured Progressive Knowledge Activation for LLM-Driven Neural Architecture Search](https://mlsanity.com/paper/2605.04057/) (2026)
- [QM-ToT: A Medical Tree of Thoughts Reasoning Framework for Quantized Model](https://mlsanity.com/paper/2504.12334/) (2025)
- [ONNX-Net: Towards Universal Representations and Instant Performance Prediction for Neural Architectures](https://mlsanity.com/paper/2510.04938/) (2025)
- [Agentic Neural Architecture Search](https://mlsanity.com/paper/2607.07984/) (2026)
- [Beyond Average Performance: Dynamic Instance Clustering and Specialized Algorithm Design in LLM-Assisted Evolutionary Search](https://mlsanity.com/paper/2608.03129/) (2026)
- [LLM-Driven Composite Neural Architecture Search for Multi-Source RL State Encoding](https://mlsanity.com/paper/2512.06982/) (2025)
- [HERCULES: Hardware-Efficient, Robust, Continual Learning Neural Architecture Search](https://mlsanity.com/paper/2605.04103/) (2026)
- [Structuring Open-Ended NAS: Semi-Automated Design Knowledge Structuring with LLMs for Efficient Neural Architecture Search](https://mlsanity.com/paper/2605.19247/) (2026)
- [Scaling Closed-Loop Feature Channel Configuration with LLMs](https://mlsanity.com/paper/2607.20516/) (2026)
- [Bi-EZP: LLM-Guided Bilevel Program Evolution for Ensemble Zero-Cost Proxy Discovery](https://mlsanity.com/paper/2608.21927/) (2026)
- [ES-AHD: An Evolution Strategy Framework for Automatic Heuristic Design](https://mlsanity.com/paper/2609.00023/) (2026)
- [From Hand-Crafted to LLM-Based Variation Operators in Metaheuristics: A Tutorial](https://mlsanity.com/paper/2609.31649/) (2026)
- [Efficient Audio-Visual Speech Enhancement via Reinforcement Learning-Based Neural Architecture Search](https://mlsanity.com/paper/2610.12796/) (2026)
- [GeoBridge: Evolving Transferable Decoder Topologies for Geometric Vision](https://mlsanity.com/paper/2610.15280/) (2026)

## Abstract (arXiv)

Transferable neural architecture search (TNAS) has been introduced to design efficient neural architectures for multiple tasks, to enhance the practical applicability of NAS in real-world scenarios. In TNAS, architectural knowledge accumulated in previous search processes is reused to warm up the architecture search for new tasks. However, existing TNAS methods still search in an extensive search space, necessitating the evaluation of numerous architectures. To overcome this challenge, this work proposes a novel transfer paradigm, i.e., design principle transfer. In this work, the linguistic description of various structural components' effects on architectural performance is termed design principles. They are learned from established architectures and then can be reused to reduce the search space by discarding unpromising architectures. Searching in the refined search space can boost both the search performance and efficiency for new NAS tasks. To this end, a large language model (LLM)-assisted design principle transfer (LAPT) framework is devised. In LAPT, LLM is applied to automatically reason the design principles from a set of given architectures, and then a principle adaptation method is applied to refine these principles progressively based on the new search results. Experimental results show that LAPT can beat the state-of-the-art TNAS methods on most tasks and achieve comparable performance on others.