# Learning Multi-Task Gaussian Process Over Heterogeneous Input Domains

arXiv: https://arxiv.org/abs/2202.12636 · HTML: https://mlsanity.com/paper/2202.12636/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 68 references that are in the map.
- First posted on arXiv 25 Feb 2022.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [Deep Transfer Learning with Joint Adaptation Networks](https://mlsanity.com/paper/1605.06636/) (2016)
- [Gaussian Processes for Big Data](https://mlsanity.com/paper/1309.6835/) (2013)
- [When Gaussian Process Meets Big Data: A Review of Scalable GPs](https://mlsanity.com/paper/1807.01065/) (2018)
- [GPflow: A Gaussian process library using TensorFlow](https://mlsanity.com/paper/1610.08733/) (2016)
- [Generalized Robust Bayesian Committee Machine for Large-scale Gaussian Process Regression](https://mlsanity.com/paper/1806.00720/) (2018)
- [Spectral Mixture Kernels for Multi-Output Gaussian Processes](https://mlsanity.com/paper/1709.01298/) (2017)
- [Heterogeneous Multi-output Gaussian Process Prediction](https://mlsanity.com/paper/1805.07633/) (2018)
- [Sparse Gaussian Process Variational Autoencoders](https://mlsanity.com/paper/2010.10177/) (2020)
- [Distributed Variational Inference in Sparse Gaussian Process Regression and Latent Variable Models](https://mlsanity.com/paper/1402.1389/) (2014)
- [Multi-fidelity modeling with different input domain definitions using Deep Gaussian Processes](https://mlsanity.com/paper/2006.15924/) (2020)
- [Tensor Regression Meets Gaussian Processes](https://mlsanity.com/paper/1710.11345/) (2017)
- [The Gaussian Process Autoregressive Regression Model (GPAR)](https://mlsanity.com/paper/1802.07182/) (2018)
- [Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels](https://mlsanity.com/paper/1910.05199/) (2019)
- [Transferring model structure in Bayesian transfer learning for Gaussian process regression](https://mlsanity.com/paper/2101.06884/) (2021)
- [Scalable Multi-Task Gaussian Processes with Neural Embedding of Coregionalization](https://mlsanity.com/paper/2109.09261/) (2021)
- [Multi-Output Convolution Spectral Mixture for Gaussian Processes](https://mlsanity.com/paper/1808.02266/) (2018)
- [Minimax Error of Interpolation and Optimal Design of Experiments for Variable Fidelity Data](https://mlsanity.com/paper/1610.06731/) (2016)
- [Lateral Force Prediction using Gaussian Process Regression for Intelligent Tire Systems](https://mlsanity.com/paper/2009.12463/) (2020)

## Built on by (most cited first, 7 of 7)

- [Multi-fidelity surrogate with heterogeneous input spaces for modeling melt pools in laser-directed energy deposition](https://mlsanity.com/paper/2403.13136/) (2024)
- [Heterogenous Multi-Source Data Fusion Through Input Mapping and Latent Variable Gaussian Process](https://mlsanity.com/paper/2407.11268/) (2024)
- [Large Language Model as Meta-Surrogate for Data-Driven Many-Task Optimization: A Proof-of-Principle Study](https://mlsanity.com/paper/2503.08301/) (2025)
- [Exact and general decoupled solutions of the LMC Multitask Gaussian Process model](https://mlsanity.com/paper/2310.12032/) (2023)
- [R^2-HGP: A Double-Regularized Gaussian Process for Heterogeneous Transfer Learning](https://mlsanity.com/paper/2512.10258/) (2025)
- [Offline Multi-Task Multi-Objective Data-Driven Evolutionary Algorithm with Language Surrogate Model and Implicit Q-Learning](https://mlsanity.com/paper/2512.15149/) (2025)
- [Multi-task Modeling for Engineering Applications with Sparse Data](https://mlsanity.com/paper/2601.05910/) (2026)

## Abstract (arXiv)

Multi-task Gaussian process (MTGP) is a well-known non-parametric Bayesian model for learning correlated tasks effectively by transferring knowledge across tasks. But current MTGPs are usually limited to the multi-task scenario defined in the same input domain, leaving no space for tackling the heterogeneous case, i.e., the features of input domains vary over tasks. To this end, this paper presents a novel heterogeneous stochastic variational linear model of coregionalization (HSVLMC) model for simultaneously learning the tasks with varied input domains. Particularly, we develop the stochastic variational framework with Bayesian calibration that (i) takes into account the effect of dimensionality reduction raised by domain mappings in order to achieve effective input alignment; and (ii) employs a residual modeling strategy to leverage the inductive bias brought by prior domain mappings for better model inference. Finally, the superiority of the proposed model against existing LMC models has been extensively verified on diverse heterogeneous multi-task cases and a practical multi-fidelity steam turbine exhaust problem.