mlsanityData to 7 Oct 2026

AutoSpace: Neural Architecture Search with Less Human Interference

arXiv 2103.11833 ↗First posted 22 Mar 2021cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Current neural architecture search (NAS) algorithms still require expert knowledge and effort to design a search space for network construction. In this paper, we consider automating the search space design to minimize human interference, which however faces two challenges: the explosive complexity of the exploration space and the expensive computation cost to evaluate the quality of different search spaces. To solve them, we propose a novel differentiable evolutionary framework named AutoSpace, which evolves the search space to an optimal one with following novel techniques: a differentiable fitness scoring function to efficiently evaluate the performance of cells and a reference architecture to speedup the evolution procedure and avoid falling into sub-optimal solutions. The framework is generic and compatible with additional computational constraints, making it feasible to learn specialized search spaces that fit different computational budgets. With the learned search space, the performance of recent NAS algorithms can be improved significantly compared with using previously manually designed spaces. Remarkably, the models generated from the new search space achieve 77.8% top-1 accuracy on ImageNet under the mobile setting (MAdds < 500M), out-performing previous SOTA EfficientNet-B0 by 0.7%. All codes will be made public.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 39 of its 47 references that are in the map. First posted on arXiv 22 Mar 2021.

Builds on

39 of 47 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2016Densely Connected Convolutional Networkscited by 6,691
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2017Squeeze-and-Excitation Networkscited by 4,763
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  9. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  10. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  11. 2019Searching for MobileNetV3cited by 1,984
  12. 2018DARTS: Differentiable Architecture Searchcited by 1,782
27 more
  1. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  2. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  4. 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
  5. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  6. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  7. 2018AutoAugment: Learning Augmentation Policies from Datacited by 728
  8. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 673
  9. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  10. 2017Progressive Neural Architecture Searchcited by 483
  11. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  12. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  13. 2017Large-Scale Evolution of Image Classifierscited by 348
  14. 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
  15. 2021Coordinate Attention for Efficient Mobile Network Designcited by 239
  16. 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 168
  17. 2020FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensionscited by 152
  18. 2019Evaluating the Search Phase of Neural Architecture Searchcited by 120
  19. 2019MixConv: Mixed Depthwise Convolutional Kernelscited by 108
  20. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  21. 2016Asynchronous Stochastic Gradient Descent with Delay Compensationcited by 98
  22. 2018ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptationcited by 76
  23. 2019On Network Design Spaces for Visual Recognitioncited by 68
  24. 2020Rethinking Bottleneck Structure for Efficient Mobile Network Designcited by 54
  25. 2020FBNetV3: Joint Architecture-Recipe Search using Predictor Pretrainingcited by 35
  26. 2020Deeper Insights into Weight Sharing in Neural Architecture Searchcited by 24
  27. 2019Neural Epitome Search for Architecture-Agnostic Network Compressioncited by 8

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3.

5 papers.

  1. 2022Diffusion Probabilistic Model Made Slimcited by 121
  2. 2022Deep Model Reassemblycited by 78
  3. 2023OTOv3: Automatic Architecture-Agnostic Neural Network Training and Compression from Structured Pruning to Erasing Operatorscited by 6
  4. 2023Neural Fine-Tuning Search for Few-Shot Learningcited by 4
  5. 2023Automated Search-Space Generation Neural Architecture Searchcited by 0