# Manifold Gaussian Processes for Regression

arXiv: https://arxiv.org/abs/1402.5876 · HTML: https://mlsanity.com/paper/1402.5876/

## Key facts

- Cited by 80 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Feb 2014.

## Built on by (most cited first, 30 of 80)

- [Large Language Models to Enhance Bayesian Optimization](https://mlsanity.com/paper/2402.03921/) (2024)
- [Priors in Bayesian Deep Learning: A Review](https://mlsanity.com/paper/2105.06868/) (2021)
- [On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty](https://mlsanity.com/paper/2102.11409/) (2021)
- [The Promises and Pitfalls of Deep Kernel Learning](https://mlsanity.com/paper/2102.12108/) (2021)
- [Personalized Federated Learning with Gaussian Processes](https://mlsanity.com/paper/2106.15482/) (2021)
- [PFNs4BO: In-Context Learning for Bayesian Optimization](https://mlsanity.com/paper/2305.17535/) (2023)
- [A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness](https://mlsanity.com/paper/2205.00403/) (2022)
- [Invariance Learning in Deep Neural Networks with Differentiable Laplace Approximations](https://mlsanity.com/paper/2202.10638/) (2022)
- [Deep networks for system identification: a Survey](https://mlsanity.com/paper/2301.12832/) (2023)
- [GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning](https://mlsanity.com/paper/2102.07868/) (2021)
- [Last Layer Marginal Likelihood for Invariance Learning](https://mlsanity.com/paper/2106.07512/) (2021)
- [Meta-learning Adaptive Deep Kernel Gaussian Processes for Molecular Property Prediction](https://mlsanity.com/paper/2205.02708/) (2022)
- [Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoires](https://mlsanity.com/paper/2109.08522/) (2021)
- [A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems](https://mlsanity.com/paper/2202.04648/) (2022)
- [Dense Gaussian Processes for Few-Shot Segmentation](https://mlsanity.com/paper/2110.03674/) (2021)
- [Statistical Deep Learning for Spatial and Spatio-Temporal Data](https://mlsanity.com/paper/2206.02218/) (2022)
- [Predictive Churn with the Set of Good Models](https://mlsanity.com/paper/2402.07745/) (2024)
- [Constraining Gaussian Processes to Systems of Linear Ordinary Differential Equations](https://mlsanity.com/paper/2208.12515/) (2022)
- [Kernel Identification Through Transformers](https://mlsanity.com/paper/2106.08185/) (2021)
- [Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learning](https://mlsanity.com/paper/2402.04005/) (2024)
- [JUMBO: Scalable Multi-task Bayesian Optimization using Offline Data](https://mlsanity.com/paper/2106.00942/) (2021)
- [Scalable Gaussian-process regression and variable selection using Vecchia approximations](https://mlsanity.com/paper/2202.12981/) (2022)
- [Efficient Transformed Gaussian Processes for Non-Stationary Dependent Multi-class Classification](https://mlsanity.com/paper/2205.15008/) (2022)
- [DKL-KAN: Scalable Deep Kernel Learning using Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2407.21176/) (2024)
- [Maximum Likelihood Uncertainty Estimation: Robustness to Outliers](https://mlsanity.com/paper/2202.03870/) (2022)
- [Deep Kernel Learning of Dynamical Models from High-Dimensional Noisy Data](https://mlsanity.com/paper/2208.12975/) (2022)
- [Guided Deep Kernel Learning](https://mlsanity.com/paper/2302.09574/) (2023)
- [Bayesian Exploration of Pre-trained Models for Low-shot Image Classification](https://mlsanity.com/paper/2404.00312/) (2024)
- [Federated Bayesian Neural Regression: A Scalable Global Federated Gaussian Process](https://mlsanity.com/paper/2206.06357/) (2022)
- [Emulating the dynamics of complex systems using autoregressive models on manifolds (mNARX)](https://mlsanity.com/paper/2306.16335/) (2023)

## Abstract (arXiv)

Off-the-shelf Gaussian Process (GP) covariance functions encode smoothness assumptions on the structure of the function to be modeled. To model complex and non-differentiable functions, these smoothness assumptions are often too restrictive. One way to alleviate this limitation is to find a different representation of the data by introducing a feature space. This feature space is often learned in an unsupervised way, which might lead to data representations that are not useful for the overall regression task. In this paper, we propose Manifold Gaussian Processes, a novel supervised method that jointly learns a transformation of the data into a feature space and a GP regression from the feature space to observed space. The Manifold GP is a full GP and allows to learn data representations, which are useful for the overall regression task. As a proof-of-concept, we evaluate our approach on complex non-smooth functions where standard GPs perform poorly, such as step functions and robotics tasks with contacts.