A Survey on Multi-Objective Neural Architecture Search
Not on the map. Why?
Abstract (arXiv)
Recently, the expert-crafted neural architectures is increasing overtaken by the utilization of neural architecture search (NAS) and automatic generation (and tuning) of network structures which has a close relation to the Hyperparameter Optimization and Auto Machine Learning (AutoML). After the earlier NAS attempts to optimize only the prediction accuracy, Multi-Objective Neural architecture Search (MONAS) has been attracting attentions which considers more goals such as computational complexity, power consumption, and size of the network for optimization, reaching a trade-off between the accuracy and other features like the computational cost. In this paper, we present an overview of principal and state-of-the-art works in the field of MONAS. Starting from a well-categorized taxonomy and formulation for the NAS, we address and correct some miscategorizations in previous surveys of the NAS field. We also provide a list of all known objectives used and add a number of new ones and elaborate their specifications. We have provides analyses about the most important objectives and shown that the stochastic properties of some the them should be differed from deterministic ones in the multi-objective optimization procedure of NAS. We finalize this paper with a number of future directions and topics in the field of MONAS.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 74 of its 113 references that are in the map. First posted on arXiv 18 Jul 2023.
Builds on
74 of 113 references.
References · 74
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
- 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
- 2019Searching for MobileNetV3cited by 1,984
- 2018DARTS: Differentiable Architecture Searchcited by 1,782
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
- 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
- 2021Transformers in Vision: A Surveycited by 1,009
- 2018MnasNet: Platform-Aware Neural Architecture Search for Mobilecited by 937
- 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
38 more
- 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
- 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
- 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
- 2017Progressive Neural Architecture Searchcited by 483
- 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
- 2020NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Searchcited by 412
- 2019AutoML: A Survey of the State-of-the-Artcited by 388
- 2017Large-Scale Evolution of Image Classifierscited by 348
- 2019NAS-Bench-101: Towards Reproducible Neural Architecture Searchcited by 340
- 2018SNAS: Stochastic Neural Architecture Searchcited by 288
- 2016Designing Neural Network Architectures using Reinforcement Learningcited by 277
- 2019Random Search and Reproducibility for Neural Architecture Searchcited by 253
- 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
- 2020Neural Architecture Search without Trainingcited by 241
- 2019Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluationcited by 230
- 2018Neural Architecture Search with Bayesian Optimisation and Optimal Transportcited by 192
- 2020Hierarchical Neural Architecture Search for Deep Stereo Matchingcited by 166
- 2018Efficient Multi-objective Neural Architecture Search via Lamarckian Evolutioncited by 151
- 2020A Survey on Evolutionary Neural Architecture Searchcited by 142
- 2019Exploring Randomly Wired Neural Networks for Image Recognitioncited by 103
- 2017Evolving Deep Neural Networkscited by 93
- 2021LightTrack: Finding Lightweight Neural Networks for Object Tracking via One-Shot Architecture Searchcited by 82
- 2019CARS: Continuous Evolution for Efficient Neural Architecture Searchcited by 79
- 2020NSGANetV2: Evolutionary Multi-Objective Surrogate-Assisted Neural Architecture Searchcited by 75
- 2019A Survey on Neural Architecture Searchcited by 71
- 2019Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Searchcited by 66
- 2017A Genetic Programming Approach to Designing Convolutional Neural Network Architecturescited by 59
- 2020MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulationcited by 56
- 2019Best Practices for Scientific Research on Neural Architecture Searchcited by 53
- 2019Densely Connected Search Space for More Flexible Neural Architecture Searchcited by 52
- 2018DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architecturescited by 40
- 2019Partial Order Pruning: for Best Speed/Accuracy Trade-off in Neural Architecture Searchcited by 39
- 2022PaSca: a Graph Neural Architecture Search System under the Scalable Paradigmcited by 37
- 2020Neuroevolution in Deep Neural Networks: Current Trends and Future Challengescited by 33
- 2018BlockQNN: Efficient Block-wise Neural Network Architecture Generationcited by 31
- 2022Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterizationcited by 31
- 2020Best of Both Worlds: AutoML Codesign of a CNN and its Hardware Acceleratorcited by 30
- 2018Learning to Design RNAcited by 27
50 most cited of 74 All 74 (JSON)
Built on by
Not cited yet.