# Exact and general decoupled solutions of the LMC Multitask Gaussian Process model

arXiv: https://arxiv.org/abs/2310.12032 · HTML: https://mlsanity.com/paper/2310.12032/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 47 references that are in the map.
- First posted on arXiv 18 Oct 2023.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Scalable Variational Gaussian Process Classification](https://mlsanity.com/paper/1411.2005/) (2014)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [Cheap Orthogonal Constraints in Neural Networks: A Simple Parametrization of the Orthogonal and Unitary Group](https://mlsanity.com/paper/1901.08428/) (2019)
- [Gaussian Process Kernels for Pattern Discovery and Extrapolation](https://mlsanity.com/paper/1302.4245/) (2013)
- [Orthogonal Recurrent Neural Networks with Scaled Cayley Transform](https://mlsanity.com/paper/1707.09520/) (2017)
- [Trivializations for Gradient-Based Optimization on Manifolds](https://mlsanity.com/paper/1909.09501/) (2019)
- [Recent Advances in Bayesian Optimization](https://mlsanity.com/paper/2206.03301/) (2022)
- [Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Trees](https://mlsanity.com/paper/2006.15779/) (2020)
- [Gaussian process regression for forecasting battery state of health](https://mlsanity.com/paper/1703.05687/) (2017)
- [Scalable Exact Inference in Multi-Output Gaussian Processes](https://mlsanity.com/paper/1911.06287/) (2019)
- [Conditioning Sparse Variational Gaussian Processes for Online Decision-making](https://mlsanity.com/paper/2110.15172/) (2021)
- [Bayesian Multi-Task Learning MPC for Robotic Mobile Manipulation](https://mlsanity.com/paper/2211.10270/) (2022)
- [Near-Optimal Active Learning of Multi-Output Gaussian Processes](https://mlsanity.com/paper/1511.06891/) (2015)
- [Learning Multi-Task Gaussian Process Over Heterogeneous Input Domains](https://mlsanity.com/paper/2202.12636/) (2022)
- [Generalized probabilistic principal component analysis of correlated data](https://mlsanity.com/paper/1808.10868/) (2018)
- [Scalable Multi-Task Gaussian Processes with Neural Embedding of Coregionalization](https://mlsanity.com/paper/2109.09261/) (2021)
- [Generic Inference in Latent Gaussian Process Models](https://mlsanity.com/paper/1609.00577/) (2016)
- [Grouped Gaussian Processes for Solar Power Prediction](https://mlsanity.com/paper/1806.02543/) (2018)
- [QR and LQ Decomposition Matrix Backpropagation Algorithms for Square, Wide, and Deep -- Real or Complex -- Matrices and Their Software Implementation](https://mlsanity.com/paper/2009.10071/) (2020)

## Abstract (arXiv)

The Linear Model of Co-regionalization (LMC) is a very general multitask gaussian process model for regression or classification. While its expressiveness and conceptual simplicity are appealing, naive implementations have cubic complexity in the product (number of datapoints × number of tasks), making approximations mandatory for most applications. However, recent work has shown that in some settings the latent processes of the model can be decoupled, leading to a complexity that is only linear in the number of said processes. We here extend these results, showing from the most general assumptions that the only condition necessary to an efficient exact computation of the LMC is a mild hypothesis on the noise model. We introduce a full parametrization of the resulting projected LMC model, enabling its efficient optimization. The effectiveness of this approach is assessed through synthetic and real-data experiments, testing in particular the behavior of its underlying noise model restriction. Overall, the projected LMC appears as a competitive and simpler alternative to state-of-the art multitask gaussian process models. It greatly facilitates some computations such as training data updates or leave-one-out cross-validation, and is more interpretable, for it gives access to its low-dimensional quantities and to their explicit relation with the full-dimensional data. These qualities could facilitate the adoption by various industries of entire classes of methodologies, notably multitask bayesian optimization.