# Zero-Shot Task Transfer

arXiv: https://arxiv.org/abs/1903.01092 · HTML: https://mlsanity.com/paper/1903.01092/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2019.

## Built on by (most cited first, 26 of 26)

- [MulT: An End-to-End Multitask Learning Transformer](https://mlsanity.com/paper/2205.08303/) (2022)
- [Task Aligned Generative Meta-learning for Zero-shot Learning](https://mlsanity.com/paper/2103.02185/) (2021)
- [Egocentric Video Task Translation](https://mlsanity.com/paper/2212.06301/) (2022)
- [Generative Modeling for Multi-task Visual Learning](https://mlsanity.com/paper/2106.13409/) (2021)
- [Vision Transformer Adapters for Generalizable Multitask Learning](https://mlsanity.com/paper/2308.12372/) (2023)
- [A review and experimental evaluation of deep learning methods for MRI reconstruction](https://mlsanity.com/paper/2109.08618/) (2021)
- [FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialogue](https://mlsanity.com/paper/2205.06262/) (2022)
- [Attribute-Modulated Generative Meta Learning for Zero-Shot Classification](https://mlsanity.com/paper/2104.10857/) (2021)
- [Task Affinity with Maximum Bipartite Matching in Few-Shot Learning](https://mlsanity.com/paper/2110.02399/) (2021)
- [Evaluating the structure of cognitive tasks with transfer learning](https://mlsanity.com/paper/2308.02408/) (2023)
- [Fisher Task Distance and Its Application in Neural Architecture Search](https://mlsanity.com/paper/2103.12827/) (2021)
- [Zero-Shot Medical Phrase Grounding with Off-the-shelf Diffusion Models](https://mlsanity.com/paper/2404.12920/) (2024)
- [Time Series Predictions in Unmonitored Sites: A Survey of Machine Learning Techniques in Water Resources](https://mlsanity.com/paper/2308.09766/) (2023)
- [Transfer Learning for Individual Treatment Effect Estimation](https://mlsanity.com/paper/2210.00380/) (2022)
- [Diversity-boosted Generalization-Specialization Balancing for Zero-shot Learning](https://mlsanity.com/paper/2201.01961/) (2022)
- [Task Grouping for Automated Multi-Task Machine Learning via Task Affinity Prediction](https://mlsanity.com/paper/2310.16241/) (2023)
- [Learning Good Features to Transfer Across Tasks and Domains](https://mlsanity.com/paper/2301.11310/) (2023)
- [Mode-Aware Continual Learning for Conditional Generative Adversarial Networks](https://mlsanity.com/paper/2305.11400/) (2023)
- [Enhancing Monocular Depth Estimation with Multi-Source Auxiliary Tasks](https://mlsanity.com/paper/2501.12824/) (2025)
- [Learning Invariant Representations across Domains and Tasks](https://mlsanity.com/paper/2103.05114/) (2021)
- [T2T-VICL: Cross-Task Visual In-Context Learning via Implicit Text-Driven VLMs](https://mlsanity.com/paper/2511.16107/) (2025)
- [Multi-Task Neural Processes](https://mlsanity.com/paper/2110.14953/) (2021)
- [Towards Estimating Transferability using Hard Subsets](https://mlsanity.com/paper/2301.06928/) (2023)
- [Ensemble Prediction of Task Affinity for Efficient Multi-Task Learning](https://mlsanity.com/paper/2602.18591/) (2026)
- [Hidden-Shot: Towards One-Shot Task Generalization for Low-Level Vision Generalist Models](https://mlsanity.com/paper/2607.01535/) (2026)
- [Geometric Similarity in VLM Low-Level Vision Representations](https://mlsanity.com/paper/2610.00848/) (2026)

## Abstract (arXiv)

In this work, we present a novel meta-learning algorithm, i.e. TTNet, that regresses model parameters for novel tasks for which no ground truth is available (zero-shot tasks). In order to adapt to novel zero-shot tasks, our meta-learner learns from the model parameters of known tasks (with ground truth) and the correlation of known tasks to zero-shot tasks. Such intuition finds its foothold in cognitive science, where a subject (human baby) can adapt to a novel-concept (depth understanding) by correlating it with old concepts (hand movement or self-motion), without receiving explicit supervision. We evaluated our model on the Taskonomy dataset, with four tasks as zero-shot: surface-normal, room layout, depth, and camera pose estimation. These tasks were chosen based on the data acquisition complexity and the complexity associated with the learning process using a deep network. Our proposed methodology out-performs state-of-the-art models (which use ground truth)on each of our zero-shot tasks, showing promise on zero-shot task transfer. We also conducted extensive experiments to study the various choices of our methodology, as well as showed how the proposed method can also be used in transfer learning. To the best of our knowledge, this is the firstsuch effort on zero-shot learning in the task space.