# Multi-Task Neural Processes

arXiv: https://arxiv.org/abs/2111.05820 · HTML: https://mlsanity.com/paper/2111.05820/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 53 references that are in the map.
- First posted on arXiv 10 Nov 2021.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Gradient Surgery for Multi-Task Learning](https://mlsanity.com/paper/2001.06782/) (2020)
- [Deep Hashing Network for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1706.07522/) (2017)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [End-to-End Multi-Task Learning with Attention](https://mlsanity.com/paper/1803.10704/) (2018)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Conditional Neural Processes](https://mlsanity.com/paper/1807.01613/) (2018)
- [Neural Processes](https://mlsanity.com/paper/1807.01622/) (2018)
- [Attentive Neural Processes](https://mlsanity.com/paper/1901.05761/) (2019)
- [AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning](https://mlsanity.com/paper/1911.12423/) (2019)
- [Convolutional Conditional Neural Processes](https://mlsanity.com/paper/1910.13556/) (2019)
- [Fast and Flexible Multi-Task Classification Using Conditional Neural Adaptive Processes](https://mlsanity.com/paper/1906.07697/) (2019)
- [Learning to Branch for Multi-Task Learning](https://mlsanity.com/paper/2006.01895/) (2020)
- [Hierarchical Variational Models](https://mlsanity.com/paper/1511.02386/) (2015)
- [Association of genomic subtypes of lower-grade gliomas with shape features automatically extracted by a deep learning algorithm](https://mlsanity.com/paper/1906.03720/) (2019)
- [Bayesian Hypernetworks](https://mlsanity.com/paper/1710.04759/) (2017)
- [Learning Hierarchical Priors in VAEs](https://mlsanity.com/paper/1905.04982/) (2019)
- [Doubly Stochastic Variational Inference for Neural Processes with Hierarchical Latent Variables](https://mlsanity.com/paper/2008.09469/) (2020)
- [When Age-Invariant Face Recognition Meets Face Age Synthesis: A Multi-Task Learning Framework](https://mlsanity.com/paper/2103.01520/) (2021)
- [Learning Hierarchical Features from Generative Models](https://mlsanity.com/paper/1702.08396/) (2017)
- [Deep Multi-Task Learning for Joint Localization, Perception, and Prediction](https://mlsanity.com/paper/2101.06720/) (2021)
- [Multi-task Learning for Aggregated Data using Gaussian Processes](https://mlsanity.com/paper/1906.09412/) (2019)
- [Learning to Learn Variational Semantic Memory](https://mlsanity.com/paper/2010.10341/) (2020)

## Built on by (most cited first, 7 of 7)

- [The Neural Process Family: Survey, Applications and Perspectives](https://mlsanity.com/paper/2209.00517/) (2022)
- [Amortized Probabilistic Conditioning for Optimization, Simulation and Inference](https://mlsanity.com/paper/2410.15320/) (2024)
- [In-Context In-Context Learning with Transformer Neural Processes](https://mlsanity.com/paper/2406.13493/) (2024)
- [Task Aware Modulation using Representation Learning: An Approach for Few Shot Learning in Environmental Systems](https://mlsanity.com/paper/2310.04727/) (2023)
- [Probabilistic Interactive 3D Segmentation with Hierarchical Neural Processes](https://mlsanity.com/paper/2505.01726/) (2025)
- [R'enyi Neural Processes](https://mlsanity.com/paper/2405.15991/) (2024)
- [Multi-task Neural Diffusion Processes](https://mlsanity.com/paper/2510.03419/) (2025)

## Abstract (arXiv)

Neural processes have recently emerged as a class of powerful neural latent variable models that combine the strengths of neural networks and stochastic processes. As they can encode contextual data in the network's function space, they offer a new way to model task relatedness in multi-task learning. To study its potential, we develop multi-task neural processes, a new variant of neural processes for multi-task learning. In particular, we propose to explore transferable knowledge from related tasks in the function space to provide inductive bias for improving each individual task. To do so, we derive the function priors in a hierarchical Bayesian inference framework, which enables each task to incorporate the shared knowledge provided by related tasks into its context of the prediction function. Our multi-task neural processes methodologically expand the scope of vanilla neural processes and provide a new way of exploring task relatedness in function spaces for multi-task learning. The proposed multi-task neural processes are capable of learning multiple tasks with limited labeled data and in the presence of domain shift. We perform extensive experimental evaluations on several benchmarks for the multi-task regression and classification tasks. The results demonstrate the effectiveness of multi-task neural processes in transferring useful knowledge among tasks for multi-task learning and superior performance in multi-task classification and brain image segmentation.