# Two-stage architectural fine-tuning with neural architecture search using early-stopping in image classification

arXiv: https://arxiv.org/abs/2202.08604 · HTML: https://mlsanity.com/paper/2202.08604/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 23 references that are in the map.
- First posted on arXiv 17 Feb 2022.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Learning Transferable Features with Deep Adaptation Networks](https://mlsanity.com/paper/1502.02791/) (2015)
- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Do Better ImageNet Models Transfer Better?](https://mlsanity.com/paper/1805.08974/) (2018)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [Random Search and Reproducibility for Neural Architecture Search](https://mlsanity.com/paper/1902.07638/) (2019)
- [Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications](https://mlsanity.com/paper/2008.02608/) (2020)
- [Neural Architecture Transfer](https://mlsanity.com/paper/2005.05859/) (2020)
- [Joint Superposition Coding and Training for Federated Learning over Multi-Width Neural Networks](https://mlsanity.com/paper/2112.02543/) (2021)
- [Boosting Deep Transfer Learning for COVID-19 Classification](https://mlsanity.com/paper/2102.08085/) (2021)
- [Trends in Neural Architecture Search: Towards the Acceleration of Search](https://mlsanity.com/paper/2108.08474/) (2021)
- [Efficient Transfer Learning via Joint Adaptation of Network Architecture and Weight](https://mlsanity.com/paper/2105.08994/) (2021)
- [Delay-Sensitive and Power-Efficient Quality Control of Dynamic Video Streaming using Adaptive Super-Resolution](https://mlsanity.com/paper/2110.05783/) (2021)

## Built on by (most cited first, 2 of 2)

- [Search Space Adaptation for Differentiable Neural Architecture Search in Image Classification](https://mlsanity.com/paper/2206.02098/) (2022)
- [Improving Reliability of Fine-tuning with Block-wise Optimisation](https://mlsanity.com/paper/2301.06133/) (2023)

## Abstract (arXiv)

In many deep neural network (DNN) applications, the difficulty of gathering high-quality data in the industry field hinders the practical use of DNN. Thus, the concept of transfer learning has emerged, which leverages the pretrained knowledge of DNNs trained on large-scale datasets. Therefore, this paper suggests two-stage architectural fine-tuning, inspired by neural architecture search (NAS). One of main ideas is mutation, which reduces the search cost using given architectural information. Moreover, early-stopping is considered which cuts NAS costs by terminating the search process in advance. Experimental results verify our proposed method reduces 32.4% computational and 22.3% searching costs.