# Fast Task-Aware Architecture Inference

arXiv: https://arxiv.org/abs/1902.05781 · HTML: https://mlsanity.com/paper/1902.05781/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2019.

## Built on by (most cited first, 7 of 7)

- [An Evolutionary Approach to Dynamic Introduction of Tasks in Large-scale Multitask Learning Systems](https://mlsanity.com/paper/2205.12755/) (2022)
- [muNet: Evolving Pretrained Deep Neural Networks into Scalable Auto-tuning Multitask Systems](https://mlsanity.com/paper/2205.10937/) (2022)
- [A Continual Development Methodology for Large-scale Multitask Dynamic ML Systems](https://mlsanity.com/paper/2209.07326/) (2022)
- [FNAS: Uncertainty-Aware Fast Neural Architecture Search](https://mlsanity.com/paper/2105.11694/) (2021)
- [Privileged Zero-Shot AutoML](https://mlsanity.com/paper/2106.13743/) (2021)
- [A Multiagent Framework for the Asynchronous and Collaborative Extension of Multitask ML Systems](https://mlsanity.com/paper/2209.14745/) (2022)
- [Multipath agents for modular multitask ML systems](https://mlsanity.com/paper/2302.02721/) (2023)

## Abstract (arXiv)

Neural architecture search has been shown to hold great promise towards the automation of deep learning. However in spite of its potential, neural architecture search remains quite costly. To this point, we propose a novel gradient-based framework for efficient architecture search by sharing information across several tasks. We start by training many model architectures on several related (training) tasks. When a new unseen task is presented, the framework performs architecture inference in order to quickly identify a good candidate architecture, before any model is trained on the new task. At the core of our framework lies a deep value network that can predict the performance of input architectures on a task by utilizing task meta-features and the previous model training experiments performed on related tasks. We adopt a continuous parametrization of the model architecture which allows for efficient gradient-based optimization. Given a new task, an effective architecture is quickly identified by maximizing the estimated performance with respect to the model architecture parameters with simple gradient ascent. It is key to point out that our goal is to achieve reasonable performance at the lowest cost. We provide experimental results showing the effectiveness of the framework despite its high computational efficiency.