# Improved Automated Machine Learning from Transfer Learning

arXiv: https://arxiv.org/abs/2103.00241 · HTML: https://mlsanity.com/paper/2103.00241/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 22 references that are in the map.
- First posted on arXiv 27 Feb 2021.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [SNAS: Stochastic Neural Architecture Search](https://mlsanity.com/paper/1812.09926/) (2018)
- [PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search](https://mlsanity.com/paper/1907.05737/) (2019)
- [Random Search and Reproducibility for Neural Architecture Search](https://mlsanity.com/paper/1902.07638/) (2019)
- [Deep Spatial Autoencoders for Visuomotor Learning](https://mlsanity.com/paper/1509.06113/) (2015)
- [Label Efficient Learning of Transferable Representations across Domains and Tasks](https://mlsanity.com/paper/1712.00123/) (2017)
- [Task-Aware Neural Architecture Search](https://mlsanity.com/paper/2010.13962/) (2020)
- [One-Shot Neural Architecture Search via Compressive Sensing](https://mlsanity.com/paper/1906.02869/) (2019)

## Built on by (most cited first, 3 of 3)

- [Transfer Learning for Individual Treatment Effect Estimation](https://mlsanity.com/paper/2210.00380/) (2022)
- [Improving Open-Domain Dialogue Evaluation with a Causal Inference Model](https://mlsanity.com/paper/2301.13372/) (2023)
- [Mode-Aware Continual Learning for Conditional Generative Adversarial Networks](https://mlsanity.com/paper/2305.11400/) (2023)

## Abstract (arXiv)

In this paper, we propose a neural architecture search framework based on a similarity measure between some baseline tasks and a target task. We first define the notion of the task similarity based on the log-determinant of the Fisher Information matrix. Next, we compute the task similarity from each of the baseline tasks to the target task. By utilizing the relation between a target and a set of learned baseline tasks, the search space of architectures for the target task can be significantly reduced, making the discovery of the best candidates in the set of possible architectures tractable and efficient, in terms of GPU days. This method eliminates the requirement for training the networks from scratch for a given target task as well as introducing the bias in the initialization of the search space from the human domain.