# NNGPT: Rethinking AutoML with Large Language Models

arXiv: https://arxiv.org/abs/2511.20333 · HTML: https://mlsanity.com/paper/2511.20333/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 10 of its 59 references that are in the map.
- First posted on arXiv 25 Nov 2025.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 10 of 10)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [EvoPrompting: Language Models for Code-Level Neural Architecture Search](https://mlsanity.com/paper/2302.14838/) (2023)
- [ClusT3: Information Invariant Test-Time Training](https://mlsanity.com/paper/2310.12345/) (2023)
- [Designing Network Algorithms via Large Language Models](https://mlsanity.com/paper/2404.01617/) (2024)
- [Architecture-Aware Learning Curve Extrapolation via Graph Ordinary Differential Equation](https://mlsanity.com/paper/2412.15554/) (2024)
- [Mixture cure semiparametric additive hazard models under partly interval censoring -- a penalized likelihood approach](https://mlsanity.com/paper/2401.01234/) (2024)
- [Self-Programming Artificial Intelligence Using Code-Generating Language Models](https://mlsanity.com/paper/2205.00167/) (2022)
- [On The Suitability of Differential Dataflow For Datalog Interpretation In Highly Dynamic Settings](https://mlsanity.com/paper/2308.04214/) (2023)

## Built on by (most cited first, 13 of 13)

- [Enhancing LLM-Based Neural Network Generation: Few-Shot Prompting and Efficient Validation for Automated Architecture Design](https://mlsanity.com/paper/2512.24120/) (2025)
- [From Memorization to Creativity: LLM as a Designer of Novel Neural Architectures](https://mlsanity.com/paper/2601.02997/) (2026)
- [LLM as a Neural Architect: Controlled Generation of Image Captioning Models Under Strict API Contracts](https://mlsanity.com/paper/2512.14706/) (2025)
- [Resource-Efficient Iterative LLM-Based NAS with Feedback Memory](https://mlsanity.com/paper/2603.12091/) (2026)
- [Closed-Loop LLM Discovery of Non-Standard Channel Priors in Vision Models](https://mlsanity.com/paper/2601.08517/) (2026)
- [Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs](https://mlsanity.com/paper/2605.04903/) (2026)
- [LEMUR 2: Unlocking Neural Network Diversity for AI](https://mlsanity.com/paper/2607.06839/) (2026)
- [Systematic Exploration of 4-Expert Heterogeneous Mixture-of-Experts via Automated Pipeline Search](https://mlsanity.com/paper/2606.23739/) (2026)
- [From Code to Prediction: Fine-Tuning LLMs for Neural Network Performance Classification in NNGPT](https://mlsanity.com/paper/2605.03686/) (2026)
- [Towards Robust Training in NNGPT AutoML Pipeline: A Loss-Optimizer Pairing Selection Study](https://mlsanity.com/paper/2606.20933/) (2026)
- [Systematic Evaluation of Learning Rate Scheduling Strategies Across Heterogeneous Architectures](https://mlsanity.com/paper/2607.08511/) (2026)
- [Similarity-Guided Curriculum Fine-Tuning of LLMs for Neural Architecture Synthesis](https://mlsanity.com/paper/2607.11591/) (2026)
- [Device-First Feedback: Toward Mobile-Native LLM-Driven Neural Architecture Search](https://mlsanity.com/paper/2608.00078/) (2026)

## Abstract (arXiv)

Building self-improving AI systems remains a fundamental challenge in the AI domain. We present NNGPT, an open-source framework that turns a large language model (LLM) into a self-improving AutoML engine for neural network development, primarily for computer vision. Unlike previous frameworks, NNGPT extends the dataset of neural networks by generating new models, enabling continuous fine-tuning of LLMs based on closed-loop system of generation, assessment, and self-improvement. It integrates within one unified workflow five synergistic LLM-based pipelines: zero-shot architecture synthesis, hyperparameter optimization (HPO), code-aware accuracy/early-stop prediction, retrieval-augmented synthesis of scope-closed PyTorch blocks (NN-RAG), and reinforcement learning. Built on the LEMUR dataset as an audited corpus with reproducible metrics, NNGPT emits from a single prompt and validates network architecture, preprocessing code, and hyperparameters, executes them end-to-end, and learns from result. The PyTorch adapter makes NNGPT framework-agnostic, enabling strong performance: NN-RAG achieves 73% executability on 1,289 targets, 3-shot prompting boosts accuracy on common datasets, and hash-based deduplication saves hundreds of runs. One-shot prediction matches search-based AutoML, reducing the need for numerous trials. HPO on LEMUR achieves RMSE 0.60, outperforming Optuna (0.64), while the code-aware predictor reaches RMSE 0.14 with Pearson r=0.78. The system has already generated over 5K validated models, proving NNGPT as an autonomous AutoML engine. Upon acceptance, the code, prompts, and checkpoints will be released for public access to enable reproducibility and facilitate community usage.