mlsanityData to 14 Oct 2026

Universal Neural Architecture Space: Covering ConvNets, Transformers and Everything in Between

arXiv 2510.06035 ↗First posted 7 Oct 2025cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

We introduce Universal Neural Architecture Space (UniNAS), a generic search space for neural architecture search (NAS) which unifies convolutional networks, transformers, and their hybrid architectures under a single, flexible framework. Our approach enables discovery of novel architectures as well as analyzing existing architectures in a common framework. We also propose a new search algorithm that allows traversing the proposed search space, and demonstrate that the space contains interesting architectures, which, when using identical training setup, outperform state-of-the-art hand-crafted architectures. Finally, a unified toolkit including a standardized training and evaluation protocol is introduced to foster reproducibility and enable fair comparison in NAS research. Overall, this work opens a pathway towards systematically exploring the full spectrum of neural architectures with a unified graph-based NAS perspective.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 44 of its 51 references that are in the map. First posted on arXiv 7 Oct 2025.

Builds on

44 of 51 references.

References · 44

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2017Mask R-CNNcited by 7,277
  6. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  8. 2017Squeeze-and-Excitation Networkscited by 4,789
  9. 2019Searching for MobileNetV3cited by 1,994
  10. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  11. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  12. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
32 more
  1. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  2. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
  3. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
  4. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 569
  5. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
  6. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  7. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  8. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  9. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
  10. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 344
  11. 2020Neural Architecture Search without Trainingcited by 244
  12. 2021Zero-Cost Proxies for Lightweight NAScited by 199
  13. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 188
  14. 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
  15. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 169
  16. 2020BRP-NAS: Prediction-based NAS using GCNscited by 111
  17. 2021How Powerful are Performance Predictors in Neural Architecture Search?cited by 90
  18. 2021Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognitioncited by 73
  19. 2019SGAS: Sequential Greedy Architecture Searchcited by 69
  20. 2022β-DARTS: Beta-Decay Regularization for Differentiable Architecture Searchcited by 64
  21. 2023ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradientscited by 58
  22. 2021Searching the Search Space of Vision Transformercited by 41
  23. 2020Are Labels Necessary for Neural Architecture Search?cited by 37
  24. 2024AZ-NAS: Assembling Zero-Cost Proxies for Network Architecture Searchcited by 28
  25. 2023DAT++: Spatially Dynamic Vision Transformer with Deformable Attentioncited by 21
  26. 2021GradSign: Model Performance Inference with Theoretical Insightscited by 19
  27. 2021Memory-Efficient Differentiable Transformer Architecture Searchcited by 13
  28. 2023Reinforce Data, Multiply Impact: Improved Model Accuracy and Robustness with Dataset Reinforcementcited by 11
  29. 2024Surprisingly Strong Performance Prediction with Neural Graph Featurescited by 11
  30. 2025L-SWAG: Layer-Sample Wise Activation with Gradients information for Zero-Shot NAS on Vision Transformerscited by 5
  31. 2023Are Neural Architecture Search Benchmarks Well Designed? A Deeper Look Into Operation Importancecited by 3
  32. 2025Training-free Neural Architecture Search through Variance of Knowledge of Deep Network Weightscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Bayesian Optimization in Sequence-to-Architecture Latent Space for Zero-Shot NAScited by 0