# Multi-Task Neural Processes

arXiv: https://arxiv.org/abs/2110.14953 · HTML: https://mlsanity.com/paper/2110.14953/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 42 references that are in the map.
- First posted on arXiv 28 Oct 2021.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Which Tasks Should Be Learned Together in Multi-task Learning?](https://mlsanity.com/paper/1905.07553/) (2019)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [Towards Fast, Accurate and Stable 3D Dense Face Alignment](https://mlsanity.com/paper/2009.09960/) (2020)
- [Face Alignment in Full Pose Range: A 3D Total Solution](https://mlsanity.com/paper/1804.01005/) (2018)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks](https://mlsanity.com/paper/1810.00825/) (2018)
- [Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes](https://mlsanity.com/paper/1906.07697/) (2019)
- [Improved Few-Shot Visual Classification](https://mlsanity.com/paper/1912.03432/) (2019)
- [Learning to Branch for Multi-Task Learning](https://mlsanity.com/paper/2006.01895/) (2020)
- [Neural ODE Processes](https://mlsanity.com/paper/2103.12413/) (2021)
- [Sequential Neural Processes](https://mlsanity.com/paper/1906.10264/) (2019)
- [Doubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variables](https://mlsanity.com/paper/2008.09469/) (2020)
- [Spectral Mixture Kernels for Multi-Output Gaussian Processes](https://mlsanity.com/paper/1709.01298/) (2017)
- [Zero-Shot Task Transfer](https://mlsanity.com/paper/1903.01092/) (2019)
- [Meta-Learning Mean Functions for Gaussian Processes](https://mlsanity.com/paper/1901.08098/) (2019)
- [Recurrent Attentive Neural Process for Sequential Data](https://mlsanity.com/paper/1910.09323/) (2019)
- [Robustifying Sequential Neural Processes](https://mlsanity.com/paper/2006.15987/) (2020)
- [Information Theoretic Meta Learning with Gaussian Processes](https://mlsanity.com/paper/2009.03228/) (2020)

## Abstract (arXiv)

Neural Processes (NPs) consider a task as a function realized from a stochastic process and flexibly adapt to unseen tasks through inference on functions. However, naive NPs can model data from only a single stochastic process and are designed to infer each task independently. Since many real-world data represent a set of correlated tasks from multiple sources (e.g., multiple attributes and multi-sensor data), it is beneficial to infer them jointly and exploit the underlying correlation to improve the predictive performance. To this end, we propose Multi-Task Neural Processes (MTNPs), an extension of NPs designed to jointly infer tasks realized from multiple stochastic processes. We build MTNPs in a hierarchical way such that inter-task correlation is considered by conditioning all per-task latent variables on a single global latent variable. In addition, we further design our MTNPs so that they can address multi-task settings with incomplete data (i.e., not all tasks share the same set of input points), which has high practical demands in various applications. Experiments demonstrate that MTNPs can successfully model multiple tasks jointly by discovering and exploiting their correlations in various real-world data such as time series of weather attributes and pixel-aligned visual modalities. We release our code at https://github.com/GitGyun/multi_task_neural_processes.