mlsanityData to 14 Oct 2026

Siamese-NAS: Using Trained Samples Efficiently to Find Lightweight Neural Architecture by Prior Knowledge

arXiv 2210.00546 ↗First posted 2 Oct 2022cs.CV cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

In the past decade, many architectures of convolution neural networks were designed by handcraft, such as Vgg16, ResNet, DenseNet, etc. They all achieve state-of-the-art level on different tasks in their time. However, it still relies on human intuition and experience, and it also takes so much time consumption for trial and error. Neural Architecture Search (NAS) focused on this issue. In recent works, the Neural Predictor has significantly improved with few training architectures as training samples. However, the sampling efficiency is already considerable. In this paper, our proposed Siamese-Predictor is inspired by past works of predictor-based NAS. It is constructed with the proposed Estimation Code, which is the prior knowledge about the training procedure. The proposed Siamese-Predictor gets significant benefits from this idea. This idea causes it to surpass the current SOTA predictor on NASBench-201. In order to explore the impact of the Estimation Code, we analyze the relationship between it and accuracy. We also propose the search space Tiny-NanoBench for lightweight CNN architecture. This well-designed search space is easier to find better architecture with few FLOPs than NASBench-201. In summary, the proposed Siamese-Predictor is a predictor-based NAS. It achieves the SOTA level, especially with limited computation budgets. It applied to the proposed Tiny-NanoBench can just use a few trained samples to find extremely lightweight CNN architecture.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 31 references that are in the map. First posted on arXiv 2 Oct 2022.

Builds on

25 of 31 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  6. 2016Densely Connected Convolutional Networkscited by 6,720
  7. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  9. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  10. 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
  11. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 909
  12. 2018SNIP: Single-shot Network Pruning based on Connection Sensitivitycited by 812
13 more
  1. 2017Progressive Neural Architecture Searchcited by 487
  2. 2020Pruning neural networks without any data by iteratively conserving synaptic flowcited by 479
  3. 2020Picking Winning Tickets Before Training by Preserving Gradient Flowcited by 471
  4. 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
  5. 2019CSPNet: A New Backbone that can Enhance Learning Capability of CNNcited by 381
  6. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
  7. 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 344
  8. 2016Designing Neural Network Architectures using Reinforcement Learningcited by 279
  9. 2021Zero-Cost Proxies for Lightweight NAScited by 199
  10. 2018Neural Architecture Optimizationcited by 167
  11. 2019Neural Predictor for Neural Architecture Searchcited by 118
  12. 2020BRP-NAS: Prediction-based NAS using GCNscited by 111
  13. 2021CSL-YOLO: A New Lightweight Object Detection System for Edge Computingcited by 3

Built on by

Not cited yet.