mlsanityData to 14 Oct 2026

Towards Self-supervised and Weight-preserving Neural Architecture Search

arXiv 2206.04125 ↗First posted 8 Jun 2022cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Neural architecture search (NAS) algorithms save tremendous labor from human experts. Recent advancements further reduce the computational overhead to an affordable level. However, it is still cumbersome to deploy the NAS techniques in real-world applications due to the fussy procedures and the supervised learning paradigm. In this work, we propose the self-supervised and weight-preserving neural architecture search (SSWP-NAS) as an extension of the current NAS framework by allowing the self-supervision and retaining the concomitant weights discovered during the search stage. As such, we simplify the workflow of NAS to a one-stage and proxy-free procedure. Experiments show that the architectures searched by the proposed framework achieve state-of-the-art accuracy on CIFAR-10, CIFAR-100, and ImageNet datasets without using manual labels. Moreover, we show that employing the concomitant weights as initialization consistently outperforms the random initialization and the two-stage weight pre-training method by a clear margin under semi-supervised learning scenarios. Codes are publicly available at https://github.com/LzVv123456/SSWP-NAS.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 45 of its 52 references that are in the map. First posted on arXiv 8 Jun 2022.

Builds on

45 of 52 references.

References · 45

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2016Densely Connected Convolutional Networkscited by 6,720
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  10. 2019Searching for MobileNetV3cited by 1,994
  11. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  12. 2018DARTS: Differentiable Architecture Searchcited by 1,798
33 more
  1. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,628
  2. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  3. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  4. 2019Contrastive Multiview Codingcited by 1,301
  5. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  6. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  7. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  8. 2016An overview of gradient descent optimization algorithmscited by 1,033
  9. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  10. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 837
  11. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  12. 2020Prototypical Contrastive Learning of Unsupervised Representationscited by 640
  13. 2017Progressive Neural Architecture Searchcited by 487
  14. 2018Rethinking ImageNet Pre-trainingcited by 377
  15. 2019Transfusion: Understanding Transfer Learning for Medical Imagingcited by 368
  16. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 344
  17. 2019NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detectioncited by 291
  18. 2018SNAS: Stochastic Neural Architecture Searchcited by 289
  19. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  20. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 260
  21. 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
  22. 2016Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Predictioncited by 217
  23. 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
  24. 2017Hierarchical Representations for Efficient Architecture Searchcited by 191
  25. 2018Neural Architecture Optimizationcited by 167
  26. 2019DARTS+: Improved Differentiable Architecture Search with Early Stoppingcited by 105
  27. 2019BayesNAS: A Bayesian Approach for Neural Architecture Searchcited by 68
  28. 2019NAS-FCOS: Fast Neural Architecture Search for Object Detectioncited by 53
  29. 2019Understanding Architectures Learnt by Cell-based Neural Architecture Searchcited by 48
  30. 2019Customizable Architecture Search for Semantic Segmentationcited by 41
  31. 2020Are Labels Necessary for Neural Architecture Search?cited by 37
  32. 2021Neural Architecture Search with Random Labelscited by 33
  33. 2020Self-supervised Neural Architecture Searchcited by 7

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1.

2 papers.

  1. 2024Unsupervised Graph Neural Architecture Search with Disentangled Self-supervisioncited by 5
  2. 2023NASiam: Efficient Representation Learning using Neural Architecture Search for Siamese Networkscited by 1