# LLM as a Tool, Not an Agent: Code-Mined Tree Transformations for Neural Architecture Search

arXiv: https://arxiv.org/abs/2604.16555 · HTML: https://mlsanity.com/paper/2604.16555/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 9 of its 77 references that are in the map.
- First posted on arXiv 17 Apr 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 9 of 9)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer](https://mlsanity.com/paper/2110.02178/) (2021)
- [A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets](https://mlsanity.com/paper/1707.08819/) (2017)
- [Can GPT-4 Perform Neural Architecture Search?](https://mlsanity.com/paper/2304.10970/) (2023)
- [Lambda-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cells](https://mlsanity.com/paper/2210.07998/) (2022)
- [LeMo-NADe: Multi-Parameter Neural Architecture Discovery with LLMs](https://mlsanity.com/paper/2402.18443/) (2024)
- [CoLLM-NAS: Collaborative Large Language Models for Efficient Knowledge-Guided Neural Architecture Search](https://mlsanity.com/paper/2509.26037/) (2025)
- [SF-Mamba: Rethinking State Space Model for Vision](https://mlsanity.com/paper/2603.16423/) (2026)
- [Mixed-precision Supernet Training from Vision Foundation Models using Low Rank Adapter](https://mlsanity.com/paper/2403.20080/) (2024)

## Built on by (most cited first, 2 of 2)

- [Structuring Open-Ended NAS: Semi-Automated Design Knowledge Structuring with LLMs for Efficient Neural Architecture Search](https://mlsanity.com/paper/2605.19247/) (2026)
- [Dictionaries, Not Darwin: Set-Level Selection Beats LLM Evolution in Scientific Equation Discovery](https://mlsanity.com/paper/2607.04108/) (2026)

## Abstract (arXiv)

Neural Architecture Search (NAS) aims to automatically discover high-performing deep neural network (DNN) architectures. However, conventional algorithm-driven NAS relies on carefully hand-crafted search spaces to ensure executability, which restricts open-ended exploration. Recent coding-based agentic approaches using large language models (LLMs) reduce manual design, but current LLMs struggle to reliably generate complex, valid architectures, and their proposals are often biased toward a narrow set of patterns observed in their training data. To bridge reliable algorithmic search with powerful LLM assistance, we propose LLMasTool, a hierarchical tree-based NAS framework for stable and open-ended model evolution. Our method automatically extracts reusable modules from arbitrary source code and represents full architectures as hierarchical trees, enabling evolution through reliable tree transformations rather than code generation. At each evolution step, coarse-level planning is governed by a diversity-guided algorithm that leverages Bayesian modeling to improve exploration efficiency, while the LLM resolves the remaining degrees of freedom to ensure a meaningful evolutionary trajectory and an executable generated architecture. With this formulation, instead of fully agentic LLM approaches, our method explores diverse directions beyond the inherent biases in the LLM. Our method improves over existing NAS methods by 0.69, 1.83, and 2.68 points on CIFAR-10, CIFAR-100, and ImageNet16-120, demonstrating its effectiveness.