mlsanityData to 14 Oct 2026

LEMUR Neural Network Dataset: Towards Seamless AutoML

arXiv 2504.10552 ↗First posted 14 Apr 2025cs.LG cs.AI cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

Neural networks are the backbone of modern artificial intelligence, but designing, evaluating, and comparing them remains labor-intensive. While numerous datasets exist for training, there are few standardized collections of the models themselves. We introduce LEMUR, an open-source dataset and framework that provides a large collection of PyTorch-based neural networks across tasks such as classification, segmentation, detection, and natural language processing. Each model follows a unified template, with configurations and results stored in a structured database to ensure consistency and reproducibility. LEMUR integrates automated hyperparameter optimization via Optuna, includes statistical analysis and visualization tools, and offers an API for seamless access to performance data. The framework is extensible, allowing researchers to add new models, datasets, or metrics without breaking compatibility. By standardizing implementations and unifying evaluation, LEMUR aims to accelerate AutoML research, enable fair benchmarking, and reduce barriers to large-scale neural network experimentation. To support adoption and collaboration, LEMUR and its plugins are released under the MIT license at: https://github.com/ABrain-One/nn-dataset https://github.com/ABrain-One/nn-plots https://github.com/ABrain-One/nn-vr

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Builds on 8 of its 37 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

8 of 37 references.

References · 8

  1. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  2. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  4. 2020Improving Reproducibility in Machine Learning Research (A Report from the NeurIPS 2019 Reproducibility Program)cited by 322
  5. 2023Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Taskscited by 72
  6. 2025Optuna vs Code Llama: Are LLMs a New Paradigm for Hyperparameter Tuning?cited by 21
  7. 2024Younger: The First Dataset for Artificial Intelligence-Generated Neural Network Architecturecited by 3
  8. 2021CrossedWires: A Dataset of Syntactically Equivalent but Semantically Disparate Deep Learning Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 14.

20 papers.

Sort:

By area:

  1. 2025VIST-GPT: Ushering in the Era of Visual Storytelling with LLMs?cited by 11
  2. 2025Preparation of Fractal-Inspired Computational Architectures for Advanced Large Language Model Analysiscited by 11
  3. 2025A Retrieval-Augmented Generation Approach to Extracting Algorithmic Logic from Neural Networkscited by 11
  4. 2025AugmentGest: Can Random Data Cropping Augmentation Boost Gesture Recognition Performance?cited by 10
  5. 2025Enhancing LLM-Based Neural Network Generation: Few-Shot Prompting and Efficient Validation for Automated Architecture Designcited by 10
  6. 2026From Memorization to Creativity: LLM as a Designer of Novel Neural Architecturescited by 10
  7. 2025LLM as a Neural Architect: Controlled Generation of Image Captioning Models Under Strict API Contractscited by 9
  8. 2026Resource-Efficient Iterative LLM-Based NAS with Feedback Memorycited by 7
  9. 2026Closed-Loop LLM Discovery of Non-Standard Channel Priors in Vision Modelscited by 6
  10. 2026Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffscited by 3
  11. 2026MobileAgeNet: Lightweight Facial Age Estimation for Mobile Deploymentcited by 2
  12. 2026Real Image Denoising with Knowledge Distillation for High-Performance Mobile NPUscited by 2
8 more
  1. 2026Convergence Theory for Iterative LLM-Based Neural Architecture Search: A Parametric Cross-Entropy Framework with Closed-Form Proxy Reliabilitycited by 1
  2. 2026Systematic Exploration of 4-Expert Heterogeneous Mixture-of-Experts via Automated Pipeline Searchcited by 1
  3. 2026From Code to Prediction: Fine-Tuning LLMs for Neural Network Performance Classification in NNGPTcited by 0
  4. 2026Towards Robust Training in NNGPT AutoML Pipeline: A Loss-Optimizer Pairing Selection Studycited by 0
  5. 2026Curating Same-Family Neural Networks for LLM-Guided Model Improvement: A Controlled Case Studycited by 0
  6. 2026Systematic Evaluation of Learning Rate Scheduling Strategies Across Heterogeneous Architecturescited by 0
  7. 2026Similarity-Guided Curriculum Fine-Tuning of LLMs for Neural Architecture Synthesiscited by 0
  8. 2026Scaling Closed-Loop Feature Channel Configuration with LLMscited by 0