mlsanityData to 7 Oct 2026

NAS-Bench-360: Benchmarking Neural Architecture Search on Diverse Tasks

arXiv 2110.05668 ↗First posted 12 Oct 2021cs.CV cs.LGcited by 32

Not on the map. Why?

Abstract (arXiv)

Most existing neural architecture search (NAS) benchmarks and algorithms prioritize well-studied tasks, e.g. image classification on CIFAR or ImageNet. This makes the performance of NAS approaches in more diverse areas poorly understood. In this paper, we present NAS-Bench-360, a benchmark suite to evaluate methods on domains beyond those traditionally studied in architecture search, and use it to address the following question: do state-of-the-art NAS methods perform well on diverse tasks? To construct the benchmark, we curate ten tasks spanning a diverse array of application domains, dataset sizes, problem dimensionalities, and learning objectives. Each task is carefully chosen to interoperate with modern CNN-based search methods while possibly being far-afield from its original development domain. To speed up and reduce the cost of NAS research, for two of the tasks we release the precomputed performance of 15,625 architectures comprising a standard CNN search space. Experimentally, we show the need for more robust NAS evaluation of the kind NAS-Bench-360 enables by showing that several modern NAS procedures perform inconsistently across the ten tasks, with many catastrophically poor results. We also demonstrate how NAS-Bench-360 and its associated precomputed results will enable future scientific discoveries by testing whether several recent hypotheses promoted in the NAS literature hold on diverse tasks. NAS-Bench-360 is hosted at https://nb360.ml.cmu.edu.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 40 of its 85 references that are in the map. First posted on arXiv 12 Oct 2021.

Builds on

40 of 85 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  4. 2016Wide Residual Networkscited by 3,057
  5. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  6. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  7. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  8. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  9. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  10. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 688
  11. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  12. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
28 more
  1. 2018BOHB: Robust and Efficient Hyperparameter Optimization at Scalecited by 466
  2. 2018Spherical CNNscited by 452
  3. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
  4. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  5. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 339
  6. 2019ROCKET: Exceptionally fast and accurate time series classification using random convolutional kernelscited by 336
  7. 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
  8. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  9. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
  10. 2019Searching for A Robust Neural Architecture in Four GPU Hourscited by 258
  11. 2019Random Search and Reproducibility for Neural Architecture Searchcited by 253
  12. 2020Neural Architecture Search without Trainingcited by 241
  13. 2018A System for Massively Parallel Hyperparameter Tuningcited by 201
  14. 2021Zero-Cost Proxies for Lightweight NAScited by 194
  15. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
  16. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 168
  17. 2020AutoML-Zero: Evolving Machine Learning Algorithms From Scratchcited by 137
  18. 2019One-Shot Neural Architecture Search via Self-Evaluated Template Networkcited by 69
  19. 2020NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Searchcited by 64
  20. 2020NAS-Bench-NLP: Neural Architecture Search Benchmark for Natural Language Processingcited by 60
  21. 2021TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Searchcited by 57
  22. 2019Densely Connected Search Space for More Flexible Neural Architecture Searchcited by 52
  23. 2018Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learningcited by 46
  24. 2020HOLMES: Health OnLine Model Ensemble Serving for Deep Learning Models in Intensive Care Unitscited by 45
  25. 2020Geometry-Aware Gradient Algorithms for Neural Architecture Searchcited by 40
  26. 2021Rethinking Neural Operations for Diverse Taskscited by 18
  27. 2019deepCR: Cosmic Ray Rejection with Deep Learningcited by 11
  28. 2020sEMG Gesture Recognition with a Simple Model of Attentioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 5; 2024: 12; 2025: 7; 2026: 5.

32 papers.

Sort:

By area:

  1. 2023Neural Architecture Search: Insights from 1000 Paperscited by 136
  2. 2025MLGym: A New Framework and Benchmark for Advancing AI Research Agentscited by 111
  3. 2025CodePDE: An Inference Framework for LLM-driven PDE Solver Generationcited by 37
  4. 2023Cross-Modal Fine-Tuning: Align then Refinecited by 34
  5. 2025Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interactioncited by 34
  6. 2024Tag-LLM: Repurposing General-Purpose LLMs for Specialized Domainscited by 30
  7. 2024UPS: Efficiently Building Foundation Models for PDE Solving via Cross-Modal Adaptationcited by 27
  8. 2022Efficient Architecture Search for Diverse Taskscited by 18
  9. 2025CAT: Content-Adaptive Image Tokenizationcited by 18
  10. 2024HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Modelscited by 10
  11. 2024Learning Modality Knowledge Alignment for Cross-Modality Transfercited by 8
  12. 2024Robust NAS under adversarial training: benchmark, theory, and beyondcited by 6
20 more
  1. 2024einspace: Searching for Neural Architectures from Fundamental Operationscited by 5
  2. 2024Enhancing Cross-Modal Fine-Tuning with Gradually Intermediate Modality Generationcited by 5
  3. 2024Efficient Deep Learning Infrastructures for Embedded Computing Systems: A Comprehensive Survey and Future Envisioncited by 5
  4. 2026ARFBench: Benchmarking Time Series Question Answering Ability for Software Incident Responsecited by 5
  5. 2023On the Hyperparameter Loss Landscapes of Machine Learning Models: An Exploratory Studycited by 4
  6. 2022AutoML for Climate Change: A Call to Actioncited by 3
  7. 2022Automated Dominative Subspace Mining for Efficient Neural Architecture Searchcited by 3
  8. 2023DNArch: Learning Convolutional Neural Architectures by Backpropagationcited by 3
  9. 2023How Much Is Hidden in the NAS Benchmarks? Few-Shot Adaptation of a NAS Predictorcited by 2
  10. 2024What explains the success of cross-modal fine-tuning with ORCA?cited by 2
  11. 2025TusoAI: Agentic Optimization for Scientific Methodscited by 2
  12. 2026Rethinking Cross-Modal Fine-Tuning: Optimizing the Interaction Between Feature Alignment and Target Fittingcited by 1
  13. 2026Agentic Neural Architecture Searchcited by 1
  14. 2024MSTAR: Multi-Scale Backbone Architecture Search for Timeseries Classificationcited by 0
  15. 2024DiTMoS: Delving into Diverse Tiny-Model Selection on Microcontrollerscited by 0
  16. 2024Task Adaptation of Reinforcement Learning-based NAS Agents through Transfer Learningcited by 0
  17. 2025Variation Matters: from Mitigating to Embracing Zero-Shot NAS Ranking Function Variationcited by 0
  18. 2025confopt: A Library for Implementation and Evaluation of Gradient-based One-Shot NAS Methodscited by 0
  19. 2026YOLO-NAS-Bench: A Surrogate Benchmark with Self-Evolving Predictors for YOLO Architecture Searchcited by 0
  20. 2026NAS-Driven Hardware Accelerator Exploration for Edge AI and Quantization Effects on the Pareto Spacecited by 0