# Transfer Learning with Neural AutoML

arXiv: https://arxiv.org/abs/1803.02780 · HTML: https://mlsanity.com/paper/1803.02780/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Mar 2018.

## Built on by (most cited first, 15 of 15)

- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [Neural Architecture Search: Insights from 1000 Papers](https://mlsanity.com/paper/2301.08727/) (2023)
- [TransNAS-Bench-101: Improving Transferability and Generalizability of Cross-Task Neural Architecture Search](https://mlsanity.com/paper/2105.11871/) (2021)
- [A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse](https://mlsanity.com/paper/2112.04660/) (2021)
- [An Introduction to Bi-level Optimization: Foundations and Applications in Signal Processing and Machine Learning](https://mlsanity.com/paper/2308.00788/) (2023)
- [Local Stochastic Bilevel Optimization with Momentum-Based Variance Reduction](https://mlsanity.com/paper/2205.01608/) (2022)
- [Arch-Graph: Acyclic Architecture Relation Predictor for Task-Transferable Neural Architecture Search](https://mlsanity.com/paper/2204.05941/) (2022)
- [AutoTransfer: AutoML with Knowledge Transfer -- An Application to Graph Neural Networks](https://mlsanity.com/paper/2303.07669/) (2023)
- [Dynamically Grown Generative Adversarial Networks](https://mlsanity.com/paper/2106.08505/) (2021)
- [Privacy-preserving Online AutoML for Domain-Specific Face Detection](https://mlsanity.com/paper/2203.08399/) (2022)
- [AutoSmart: An Efficient and Automatic Machine Learning framework for Temporal Relational Data](https://mlsanity.com/paper/2109.04115/) (2021)
- [Width Transfer: On the (In)variance of Width Optimization](https://mlsanity.com/paper/2104.13255/) (2021)
- [Meta-Learning of NAS for Few-shot Learning in Medical Image Applications](https://mlsanity.com/paper/2203.08951/) (2022)
- [Learning Morphisms with Gauss-Newton Approximation for Growing Networks](https://mlsanity.com/paper/2411.05855/) (2024)
- [Evaluation of Representation Models for Text Classification with AutoML Tools](https://mlsanity.com/paper/2106.12798/) (2021)

## Abstract (arXiv)

We reduce the computational cost of Neural AutoML with transfer learning. AutoML relieves human effort by automating the design of ML algorithms. Neural AutoML has become popular for the design of deep learning architectures, however, this method has a high computation cost. To address this we propose Transfer Neural AutoML that uses knowledge from prior tasks to speed up network design. We extend RL-based architecture search methods to support parallel training on multiple tasks and then transfer the search strategy to new tasks. On language and image classification tasks, Transfer Neural AutoML reduces convergence time over single-task training by over an order of magnitude on many tasks.