# Scalable Multi-Task Gaussian Processes with Neural Embedding of Coregionalization

arXiv: https://arxiv.org/abs/2109.09261 · HTML: https://mlsanity.com/paper/2109.09261/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 67 references that are in the map.
- First posted on arXiv 20 Sep 2021.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Importance Weighting and Variational Inference](https://mlsanity.com/paper/1808.09034/) (2018)
- [Generative Models of Visually Grounded Imagination](https://mlsanity.com/paper/1705.10762/) (2017)
- [Reinterpreting Importance-Weighted Autoencoders](https://mlsanity.com/paper/1704.02916/) (2017)
- [Multi-fidelity Gaussian Process Bandit Optimisation](https://mlsanity.com/paper/1603.06288/) (2016)
- [Spectral Mixture Kernels for Multi-Output Gaussian Processes](https://mlsanity.com/paper/1709.01298/) (2017)
- [Heterogeneous Multi-output Gaussian Process Prediction](https://mlsanity.com/paper/1805.07633/) (2018)
- [Scalable Exact Inference in Multi-Output Gaussian Processes](https://mlsanity.com/paper/1911.06287/) (2019)
- [Overview of Gaussian process based multi-fidelity techniques with variable relationship between fidelities](https://mlsanity.com/paper/2006.16728/) (2020)
- [Sparse Gaussian Process Variational Autoencoders](https://mlsanity.com/paper/2010.10177/) (2020)
- [Multi-fidelity modeling with different input domain definitions using Deep Gaussian Processes](https://mlsanity.com/paper/2006.15924/) (2020)
- [Multivariate Gaussian and Student-t Process Regression for Multi-output Prediction](https://mlsanity.com/paper/1703.04455/) (2017)
- [Tensor Regression Meets Gaussian Processes](https://mlsanity.com/paper/1710.11345/) (2017)
- [The Gaussian Process Autoregressive Regression Model (GPAR)](https://mlsanity.com/paper/1802.07182/) (2018)
- [Continual Multi-task Gaussian Processes](https://mlsanity.com/paper/1911.00002/) (2019)
- [MOGPTK: The Multi-Output Gaussian Process Toolkit](https://mlsanity.com/paper/2002.03471/) (2020)
- [Understanding and Comparing Scalable Gaussian Process Regression for Big Data](https://mlsanity.com/paper/1811.01159/) (2018)
- [Deep Latent-Variable Kernel Learning](https://mlsanity.com/paper/2005.08467/) (2020)
- [Multi-Output Convolution Spectral Mixture for Gaussian Processes](https://mlsanity.com/paper/1808.02266/) (2018)
- [Neural Likelihoods for Multi-Output Gaussian Processes](https://mlsanity.com/paper/1905.13697/) (2019)
- [A Fully Natural Gradient Scheme for Improving Inference of the Heterogeneous Multi-Output Gaussian Process Model](https://mlsanity.com/paper/1911.10225/) (2019)

## Built on by (most cited first, 5 of 5)

- [Learning Multi-Task Gaussian Process Over Heterogeneous Input Domains](https://mlsanity.com/paper/2202.12636/) (2022)
- [Scalable Multi-Output Gaussian Processes with Stochastic Variational Inference](https://mlsanity.com/paper/2407.02476/) (2024)
- [Exact and general decoupled solutions of the LMC Multitask Gaussian Process model](https://mlsanity.com/paper/2310.12032/) (2023)
- [Nonparametric modeling of the composite effect of multiple nutrients on blood glucose dynamics](https://mlsanity.com/paper/2311.03129/) (2023)
- [Transformed Latent Variable Multi-Output Gaussian Processes](https://mlsanity.com/paper/2605.05133/) (2026)

## Abstract (arXiv)

Multi-task regression attempts to exploit the task similarity in order to achieve knowledge transfer across related tasks for performance improvement. The application of Gaussian process (GP) in this scenario yields the non-parametric yet informative Bayesian multi-task regression paradigm. Multi-task GP (MTGP) provides not only the prediction mean but also the associated prediction variance to quantify uncertainty, thus gaining popularity in various scenarios. The linear model of coregionalization (LMC) is a well-known MTGP paradigm which exploits the dependency of tasks through linear combination of several independent and diverse GPs. The LMC however suffers from high model complexity and limited model capability when handling complicated multi-task cases. To this end, we develop the neural embedding of coregionalization that transforms the latent GPs into a high-dimensional latent space to induce rich yet diverse behaviors. Furthermore, we use advanced variational inference as well as sparse approximation to devise a tight and compact evidence lower bound (ELBO) for higher quality of scalable model inference. Extensive numerical experiments have been conducted to verify the higher prediction quality and better generalization of our model, named NSVLMC, on various real-world multi-task datasets and the cross-fluid modeling of unsteady fluidized bed.