GeNAS: Neural Architecture Search with Better Generalization
Not on the map. Why?
Abstract (arXiv)
Neural Architecture Search (NAS) aims to automatically excavate the optimal network architecture with superior test performance. Recent neural architecture search (NAS) approaches rely on validation loss or accuracy to find the superior network for the target data. In this paper, we investigate a new neural architecture search measure for excavating architectures with better generalization. We demonstrate that the flatness of the loss surface can be a promising proxy for predicting the generalization capability of neural network architectures. We evaluate our proposed method on various search spaces, showing similar or even better performance compared to the state-of-the-art NAS methods. Notably, the resultant architecture found by flatness measure generalizes robustly to various shifts in data distribution (e.g. ImageNet-V2,-A,-O), as well as various tasks such as object detection and semantic segmentation. Code is available at https://github.com/clovaai/GeNAS.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 50 of its 67 references that are in the map. First posted on arXiv 15 May 2023.
Builds on
50 of 67 references.
References · 50
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
- 2017Squeeze-and-Excitation Networkscited by 4,789
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
- 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
- 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,161
- 2018DARTS: Differentiable Architecture Searchcited by 1,798
- 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,499
- 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,492
38 more
- 2019Natural Adversarial Examplescited by 1,408
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
- 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 943
- 2017Focal Loss for Dense Object Detectioncited by 752
- 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
- 2017Deep Mutual Learningcited by 628
- 2017Progressive Neural Architecture Searchcited by 487
- 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 464
- 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 416
- 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 414
- 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
- 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 359
- 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 292
- 2018SNAS: Stochastic Neural Architecture Searchcited by 289
- 2016Designing Neural Network Architectures using Reinforcement Learningcited by 279
- 2017Train longer, generalize better: closing the generalization gap in large batch training of neural networkscited by 279
- 2017Three Factors Influencing Minima in SGDcited by 273
- 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
- 2020Neural Architecture Search without Trainingcited by 244
- 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
- 2017SMASH: One-Shot Model Architecture Search through HyperNetworkscited by 202
- 2019Understanding and Robustifying Differentiable Architecture Searchcited by 178
- 2021Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspectivecited by 169
- 2019Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Searchcited by 131
- 2018Revisiting Small Batch Training for Deep Neural Networkscited by 102
- 2020Stabilizing Differentiable Architecture Search via Perturbation-based Regularizationcited by 102
- 2019Asymmetric Valleys: Beyond Sharp and Flat Local Minimacited by 90
- 2020CPR: Classifier-Projection Regularization for Continual Learningcited by 62
- 2019Understanding Architectures Learnt by Cell-based Neural Architecture Searchcited by 48
- 2021Neural Architecture Search with Random Labelscited by 33
- 2020Angle-based Search Space Shrinking for Neural Architecture Searchcited by 32
- 2018BlockQNN: Efficient Block-wise Neural Network Architecture Generationcited by 31
- 2020DSRNA: Differentiable Search of Robust Neural Architecturescited by 27
- 2022Shapley-NAS: Discovering Operation Contribution for Neural Architecture Searchcited by 23
- 2021Generalization Guarantees for Neural Architecture Search with Train-Validation Splitcited by 14
- 2022DropNAS: Grouped Operation Dropout for Differentiable Architecture Searchcited by 12
- 2021Neighborhood-Aware Neural Architecture Searchcited by 6
Built on by
Citing papers per year: 2024: 1; 2025: 1; 2026: 1.
3 papers.