mlsanityData to 7 Oct 2026

Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning

arXiv 2103.08157 ↗First posted 15 Mar 2021cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Neural architecture search (NAS) has fostered various fields of machine learning. Despite its prominent dedications, many have criticized the intrinsic limitations of high computational cost. We aim to ameliorate this by proposing a pretraining scheme that can be generally applied to controller-based NAS. Our method, locality-based self-supervised classification task, leverages the structural similarity of network architectures to obtain good architecture representations. We incorporate our method into neural architecture optimization (NAO) to analyze the pretrained embeddings and its effectiveness and highlight that adding metric learning loss brings a favorable impact on NAS. Our code is available at urlhttps://github.com/Multi-Objective-NAS/self-supervised-nas.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 46 of its 55 references that are in the map. First posted on arXiv 15 Mar 2021.

Builds on

46 of 55 references.

References · 46

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  3. 2018How Powerful are Graph Neural Networks?cited by 4,355
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  5. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  6. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  7. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,097
  8. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
  9. 2018CosFace: Large Margin Cosine Loss for Deep Face Recognitioncited by 968
  10. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
  11. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  12. 2017In Defense of the Triplet Loss for Person Re-Identificationcited by 784
34 more
  1. 2017SphereFace: Deep Hypersphere Embedding for Face Recognitioncited by 733
  2. 2017Progressive Neural Architecture Searchcited by 483
  3. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
  4. 2014Deep metric learning using Triplet networkcited by 421
  5. 2019Multi-Similarity Loss with General Pair Weighting for Deep Metric Learningcited by 368
  6. 2017Large-Scale Evolution of Image Classifierscited by 348
  7. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
  8. 2016Large-Margin Softmax Loss for Convolutional Neural Networkscited by 296
  9. 2019NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detectioncited by 289
  10. 2017Sampling Matters in Deep Embedding Learningcited by 285
  11. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
  12. 2018Additive Margin Softmax for Face Verificationcited by 275
  13. 2019Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentationcited by 258
  14. 2017NormFace: L2 Hypersphere Embedding for Face Verificationcited by 237
  15. 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 200
  16. 2017Hierarchical Representations for Efficient Architecture Searchcited by 188
  17. 2019The Evolved Transformercited by 180
  18. 2019BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Searchcited by 175
  19. 2018Neural Architecture Optimizationcited by 164
  20. 2019Neural Predictor for Neural Architecture Searchcited by 116
  21. 2017Deep Metric Learning with Angular Losscited by 107
  22. 2019Exploring Randomly Wired Neural Networks for Image Recognitioncited by 103
  23. 2017L2-constrained Softmax Loss for Discriminative Face Verificationcited by 96
  24. 2017Evolving Deep Neural Networkscited by 93
  25. 2019CARS: Continuous Evolution for Efficient Neural Architecture Searchcited by 79
  26. 2020Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?cited by 58
  27. 2020PyTorch Metric Learningcited by 57
  28. 2020A Generic Graph-based Neural Architecture Encoding Scheme for Predictor-based NAScited by 53
  29. 2016Learning Deep Embeddings with Histogram Losscited by 52
  30. 2020Semi-Supervised Neural Architecture Searchcited by 45
  31. 2017Simple And Efficient Architecture Search for Convolutional Neural Networkscited by 40
  32. 2020A Semi-Supervised Assessor of Neural Architecturescited by 28
  33. 2020Local Search is a Remarkably Strong Baseline for Neural Architecture Searchcited by 15
  34. 2020NASGEM: Neural Architecture Search via Graph Embedding Methodcited by 9

Built on by

Not cited yet.