# R^2-HGP: A Double-Regularized Gaussian Process for Heterogeneous Transfer Learning

arXiv: https://arxiv.org/abs/2512.10258 · HTML: https://mlsanity.com/paper/2512.10258/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 65 references that are in the map.
- First posted on arXiv 11 Dec 2025.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization](https://mlsanity.com/paper/2012.04456/) (2020)
- [A Survey on Negative Transfer](https://mlsanity.com/paper/2009.00909/) (2020)
- [Variational Auto-encoded Deep Gaussian Processes](https://mlsanity.com/paper/1511.06455/) (2015)
- [Multi-fidelity modeling with different input domain definitions using Deep Gaussian Processes](https://mlsanity.com/paper/2006.15924/) (2020)
- [Uncertainty Quantification Metrics for Deep Regression](https://mlsanity.com/paper/2405.04278/) (2024)
- [Dense Uncertainty Estimation](https://mlsanity.com/paper/2110.06427/) (2021)
- [Federated Gaussian Process: Convergence, Automatic Personalization and Multi-fidelity Modeling](https://mlsanity.com/paper/2111.14008/) (2021)
- [A Recent Survey of Heterogeneous Transfer Learning](https://mlsanity.com/paper/2310.08459/) (2023)
- [Inverse Aerodynamic Design of Gas Turbine Blades using Probabilistic Machine Learning](https://mlsanity.com/paper/2108.10163/) (2021)
- [Learning Multi-Task Gaussian Process Over Heterogeneous Input Domains](https://mlsanity.com/paper/2202.12636/) (2022)
- [Heterogeneous Multi-task Metric Learning across Multiple Domains](https://mlsanity.com/paper/1904.04081/) (2019)
- [Regularized Multi-output Gaussian Convolution Process with Domain Adaptation](https://mlsanity.com/paper/2409.02778/) (2024)
- [Gaussian Process-Gated Hierarchical Mixtures of Experts](https://mlsanity.com/paper/2302.04947/) (2023)
- [On Linear Separability under Linear Compression with Applications to Hard Support Vector Machine](https://mlsanity.com/paper/2202.01118/) (2022)
- [Heterogenous Multi-Source Data Fusion Through Input Mapping and Latent Variable Gaussian Process](https://mlsanity.com/paper/2407.11268/) (2024)

## Abstract (arXiv)

Multi-output Gaussian process (MGP) models have attracted significant attention for their flexibility and uncertainty-quantification capabilities, and have been widely adopted in multi-source transfer learning scenarios due to their ability to capture inter-task correlations. However, they still face several challenges in transfer learning. First, the input spaces of the source and target domains are often heterogeneous, which makes direct knowledge transfer difficult. Second, potential prior knowledge and physical information are typically ignored during heterogeneous transfer, hampering the utilization of domain-specific insights and leading to unstable mappings. Third, inappropriate information sharing among target and sources can easily lead to negative transfer. Traditional models fail to address these issues in a unified way. To overcome these limitations, this paper proposes a Double-Regularized Heterogeneous Gaussian Process framework (R^2-HGP). Specifically, a trainable prior probability mapping model is first proposed to align the heterogeneous input domains. The resulting aligned inputs are treated as latent variables, upon which a multi-source transfer GP model is constructed and the entire structure is integrated into a novel conditional variational autoencoder (CVAE) based framework. Physical insights is further incorporated as a regularization term to ensure that the alignment results adhere to known physical knowledge. Next, within the multi-source transfer GP model, a sparsity penalty is imposed on the transfer coefficients, enabling the model to adaptively select the most informative source outputs and suppress negative transfer. Extensive simulations and real-world engineering case studies validate the effectiveness of our R^2-HGP, demonstrating consistent superiority over state-of-the-art benchmarks across diverse evaluation metrics.