mlsanityData to 14 Oct 2026

On Network Design Spaces for Visual Recognition

arXiv 1905.13214 ↗First posted 30 May 2019cs.CV cs.LGcited by 68

Not on the map. Why?

Abstract (arXiv)

Over the past several years progress in designing better neural network architectures for visual recognition has been substantial. To help sustain this rate of progress, in this work we propose to reexamine the methodology for comparing network architectures. In particular, we introduce a new comparison paradigm of distribution estimates, in which network design spaces are compared by applying statistical techniques to populations of sampled models, while controlling for confounding factors like network complexity. Compared to current methodologies of comparing point and curve estimates of model families, distribution estimates paint a more complete picture of the entire design landscape. As a case study, we examine design spaces used in neural architecture search (NAS). We find significant statistical differences between recent NAS design space variants that have been largely overlooked. Furthermore, our analysis reveals that the design spaces for standard model families like ResNeXt can be comparable to the more complex ones used in recent NAS work. We hope these insights into distribution analysis will enable more robust progress toward discovering better networks for visual recognition.

Key facts. Cited by 68 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 27; 2022: 16; 2023: 13; 2024: 8; 2025: 3; 2026: 1.

68 papers.

Sort:

By area:

  1. 2022Scalable Diffusion Models with Transformerscited by 6,054
  2. 2022A ConvNet for the 2020scited by 3,579
  3. 2021Early Convolutions Help Transformers See Bettercited by 396
  4. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  5. 2023RepViT: Revisiting Mobile CNN From ViT Perspectivecited by 123
  6. 2021On Interaction Between Augmentations and Corruptions in Natural Corruption Robustnesscited by 97
  7. 2022Learning to Learn with Generative Models of Neural Network Checkpointscited by 86
  8. 2021Influence Selection for Active Learningcited by 84
  9. 2023Parameter-Efficient Fine-Tuning Design Spacescited by 67
  10. 2021Model Complexity of Deep Learning: A Surveycited by 65
  11. 2021Fast and Accurate Model Scalingcited by 46
  12. 2023An Unbiased Look at Datasets for Visuo-Motor Pre-Trainingcited by 42
38 more
  1. 2024Interpreting the Weight Space of Customized Diffusion Modelscited by 36
  2. 2023DisWOT: Student Architecture Search for Distillation WithOut Trainingcited by 29
  3. 2024AZ-NAS: Assembling Zero-Cost Proxies for Network Architecture Searchcited by 28
  4. 2021CompOFA: Compound Once-For-All Networks for Faster Multi-Platform Deploymentcited by 26
  5. 2022ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wildcited by 23
  6. 2022QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Explorationcited by 21
  7. 2021Fighting Gradients with Gradients: Dynamic Defenses against Adversarial Attackscited by 20
  8. 2021Trade-offs of Local SGD at Scale: An Empirical Studycited by 20
  9. 2021GradSign: Model Performance Inference with Theoretical Insightscited by 19
  10. 2022Demystifying the Neural Tangent Kernel from a Practical Perspective: Can it be trusted for Neural Architecture Search without training?cited by 18
  11. 2022PVNAS: 3D Neural Architecture Search with Point-Voxel Convolutioncited by 16
  12. 2022EZNAS: Evolving Zero Cost Proxies For Neural Architecture Scoringcited by 15
  13. 2024TinyViM: Frequency Decoupling for Tiny Hybrid Vision Mambacited by 15
  14. 2021Generic Neural Architecture Search via Regressioncited by 14
  15. 2022Generalization Properties of NAS under Activation and Skip Connection Searchcited by 14
  16. 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
  17. 2021Exploring Robust Architectures for Deep Artificial Neural Networkscited by 11
  18. 2025Regression Language Models for Codecited by 11
  19. 2021Is it enough to optimize CNN architectures on ImageNet?cited by 9
  20. 2024Encodings for Prediction-based Neural Architecture Searchcited by 9
  21. 2021Landmark Regularization: Ranking Guided Super-Net Training in Neural Architecture Searchcited by 8
  22. 2022Space4HGNN: A Novel, Modularized and Reproducible Platform to Evaluate Heterogeneous Graph Neural Networkcited by 8
  23. 2021Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metricscited by 7
  24. 2021NASOA: Towards Faster Task-oriented Online Fine-tuning with a Zoo of Modelscited by 6
  25. 2021AutoSpace: Neural Architecture Search with Less Human Interferencecited by 5
  26. 2021AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networkscited by 5
  27. 2024ParZC: Parametric Zero-Cost Proxies for Efficient NAScited by 5
  28. 2024einspace: Searching for Neural Architectures from Fundamental Operationscited by 5
  29. 2021Profiling Neural Blocks and Design Spaces for Mobile Neural Architecture Searchcited by 4
  30. 2022Privacy-preserving Online AutoML for Domain-Specific Face Detectioncited by 4
  31. 2022Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysiscited by 4
  32. 2023Neural Fine-Tuning Search for Few-Shot Learningcited by 4
  33. 2025RBFleX-NAS: Training-Free Neural Architecture Search Using Radial Basis Function Kernel and Hyperparameter Detectioncited by 4
  34. 2021Network representation learning: A macro and micro viewcited by 3
  35. 2023Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS "Cold-Start"cited by 3
  36. 2023Are Neural Architecture Search Benchmarks Well Designed? A Deeper Look Into Operation Importancecited by 3
  37. 2023Earning Extra Performance from Restrictive Feedbackscited by 3
  38. 2021An Analysis of Super-Net Heuristics in Weight-Sharing NAScited by 2

50 most cited of 68 All 68 (JSON)