# LEMUR Neural Network Dataset: Towards Seamless AutoML

arXiv: https://arxiv.org/abs/2504.10552 · HTML: https://mlsanity.com/paper/2504.10552/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 37 references that are in the map.
- First posted on arXiv 14 Apr 2025.

## Builds on (most cited first, 8 of 8)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)](https://mlsanity.com/paper/2003.12206/) (2020)
- [Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks](https://mlsanity.com/paper/2310.19909/) (2023)
- [Optuna vs Code Llama: Are LLMs a New Paradigm for Hyperparameter Tuning?](https://mlsanity.com/paper/2504.06006/) (2025)
- [Younger: The First Dataset for Artificial Intelligence-Generated Neural Network Architecture](https://mlsanity.com/paper/2406.15132/) (2024)
- [CrossedWires: A Dataset of Syntactically Equivalent but Semantically Disparate Deep Learning Models](https://mlsanity.com/paper/2108.12768/) (2021)

## Built on by (most cited first, 20 of 20)

- [VIST-GPT: Ushering in the Era of Visual Storytelling with LLMs?](https://mlsanity.com/paper/2504.19267/) (2025)
- [Preparation of Fractal-Inspired Computational Architectures for Advanced Large Language Model Analysis](https://mlsanity.com/paper/2511.07329/) (2025)
- [A Retrieval-Augmented Generation Approach to Extracting Algorithmic Logic from Neural Networks](https://mlsanity.com/paper/2512.04329/) (2025)
- [AugmentGest: Can Random Data Cropping Augmentation Boost Gesture Recognition Performance?](https://mlsanity.com/paper/2506.07216/) (2025)
- [Enhancing LLM-Based Neural Network Generation: Few-Shot Prompting and Efficient Validation for Automated Architecture Design](https://mlsanity.com/paper/2512.24120/) (2025)
- [From Memorization to Creativity: LLM as a Designer of Novel Neural Architectures](https://mlsanity.com/paper/2601.02997/) (2026)
- [LLM as a Neural Architect: Controlled Generation of Image Captioning Models Under Strict API Contracts](https://mlsanity.com/paper/2512.14706/) (2025)
- [Resource-Efficient Iterative LLM-Based NAS with Feedback Memory](https://mlsanity.com/paper/2603.12091/) (2026)
- [Closed-Loop LLM Discovery of Non-Standard Channel Priors in Vision Models](https://mlsanity.com/paper/2601.08517/) (2026)
- [Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs](https://mlsanity.com/paper/2605.04903/) (2026)
- [MobileAgeNet: Lightweight Facial Age Estimation for Mobile Deployment](https://mlsanity.com/paper/2604.17007/) (2026)
- [Real Image Denoising with Knowledge Distillation for High-Performance Mobile NPUs](https://mlsanity.com/paper/2605.03680/) (2026)
- [Convergence Theory for Iterative LLM-Based Neural Architecture Search: A Parametric Cross-Entropy Framework with Closed-Form Proxy Reliability](https://mlsanity.com/paper/2605.30103/) (2026)
- [Systematic Exploration of 4-Expert Heterogeneous Mixture-of-Experts via Automated Pipeline Search](https://mlsanity.com/paper/2606.23739/) (2026)
- [From Code to Prediction: Fine-Tuning LLMs for Neural Network Performance Classification in NNGPT](https://mlsanity.com/paper/2605.03686/) (2026)
- [Towards Robust Training in NNGPT AutoML Pipeline: A Loss-Optimizer Pairing Selection Study](https://mlsanity.com/paper/2606.20933/) (2026)
- [Curating Same-Family Neural Networks for LLM-Guided Model Improvement: A Controlled Case Study](https://mlsanity.com/paper/2607.05704/) (2026)
- [Systematic Evaluation of Learning Rate Scheduling Strategies Across Heterogeneous Architectures](https://mlsanity.com/paper/2607.08511/) (2026)
- [Similarity-Guided Curriculum Fine-Tuning of LLMs for Neural Architecture Synthesis](https://mlsanity.com/paper/2607.11591/) (2026)
- [Scaling Closed-Loop Feature Channel Configuration with LLMs](https://mlsanity.com/paper/2607.20516/) (2026)

## Abstract (arXiv)

Neural networks are the backbone of modern artificial intelligence, but designing, evaluating, and comparing them remains labor-intensive. While numerous datasets exist for training, there are few standardized collections of the models themselves. We introduce LEMUR, an open-source dataset and framework that provides a large collection of PyTorch-based neural networks across tasks such as classification, segmentation, detection, and natural language processing. Each model follows a unified template, with configurations and results stored in a structured database to ensure consistency and reproducibility. LEMUR integrates automated hyperparameter optimization via Optuna, includes statistical analysis and visualization tools, and offers an API for seamless access to performance data. The framework is extensible, allowing researchers to add new models, datasets, or metrics without breaking compatibility. By standardizing implementations and unifying evaluation, LEMUR aims to accelerate AutoML research, enable fair benchmarking, and reduce barriers to large-scale neural network experimentation. To support adoption and collaboration, LEMUR and its plugins are released under the MIT license at: https://github.com/ABrain-One/nn-dataset https://github.com/ABrain-One/nn-plots https://github.com/ABrain-One/nn-vr