# Residual Deep Gaussian Processes on Manifolds

arXiv: https://arxiv.org/abs/2411.00161 · HTML: https://mlsanity.com/paper/2411.00161/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Tutorial on Bayesian Optimization](https://mlsanity.com/paper/1807.02811/) (2018)
- [Doubly Stochastic Variational Inference for Deep Gaussian Processes](https://mlsanity.com/paper/1705.08933/) (2017)
- [Efficiently Sampling Functions from Gaussian Process Posteriors](https://mlsanity.com/paper/2002.09309/) (2020)
- [Geoopt: Riemannian Optimization in PyTorch](https://mlsanity.com/paper/2005.02819/) (2020)
- [Pymanopt: A Python Toolbox for Optimization on Manifolds using Automatic Differentiation](https://mlsanity.com/paper/1603.03236/) (2016)
- [Mat'ern Gaussian processes on Riemannian manifolds](https://mlsanity.com/paper/2006.10160/) (2020)
- [Mat'ern Gaussian Processes on Graphs](https://mlsanity.com/paper/2010.15538/) (2020)
- [Pathwise Conditioning of Gaussian Processes](https://mlsanity.com/paper/2011.04026/) (2020)
- [Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control](https://mlsanity.com/paper/1706.06491/) (2017)
- [A Framework for Interdomain and Multioutput Gaussian Processes](https://mlsanity.com/paper/2003.01115/) (2020)
- [Sparse Gaussian Processes with Spherical Harmonic Features](https://mlsanity.com/paper/2006.16649/) (2020)
- [Stationary Kernels and Gaussian Processes on Lie Groups and their Homogeneous Spaces I: the compact case](https://mlsanity.com/paper/2208.14960/) (2022)
- [Geometry-aware Bayesian Optimization in Robotics using Riemannian Mat'ern Kernels](https://mlsanity.com/paper/2111.01460/) (2021)
- [Variational Auto-encoded Deep Gaussian Processes](https://mlsanity.com/paper/1511.06455/) (2015)
- [Vector-valued Gaussian Processes on Riemannian Manifolds via Gauge Independent Projected Kernels](https://mlsanity.com/paper/2110.14423/) (2021)
- [Hodge-Compositional Edge Gaussian Processes](https://mlsanity.com/paper/2310.19450/) (2023)
- [Reproducing kernel Hilbert spaces on manifolds: Sobolev and Diffusion spaces](https://mlsanity.com/paper/1905.10913/) (2019)
- [Gaussian Process Manifold Interpolation for Probabilistic Atrial Activation Maps and Uncertain Conduction Velocity](https://mlsanity.com/paper/2004.10586/) (2020)
- [Bringing motion taxonomies to continuous domains via GPLVM on hyperbolic manifolds](https://mlsanity.com/paper/2210.01672/) (2022)
- [Recurrent Gaussian Processes](https://mlsanity.com/paper/1511.06644/) (2015)
- [Intrinsic Gaussian Vector Fields on Manifolds](https://mlsanity.com/paper/2310.18824/) (2023)
- [Isotropic Gaussian Processes on Finite Spaces of Graphs](https://mlsanity.com/paper/2211.01689/) (2022)
- [Posterior Contraction Rates for Mat'ern Gaussian Processes on Riemannian Manifolds](https://mlsanity.com/paper/2309.10918/) (2023)
- [Implicit Manifold Gaussian Process Regression](https://mlsanity.com/paper/2310.19390/) (2023)
- [Gaussian Processes and Statistical Decision-making in Non-Euclidean Spaces](https://mlsanity.com/paper/2202.10613/) (2022)

## Built on by (most cited first, 2 of 2)

- [Graph Random Features for Scalable Gaussian Processes](https://mlsanity.com/paper/2509.03691/) (2025)
- [Discrete Gaussian Vector Fields On Meshes](https://mlsanity.com/paper/2507.20024/) (2025)

## Abstract (arXiv)

We propose practical deep Gaussian process models on Riemannian manifolds, similar in spirit to residual neural networks. With manifold-to-manifold hidden layers and an arbitrary last layer, they can model manifold- and scalar-valued functions, as well as vector fields. We target data inherently supported on manifolds, which is too complex for shallow Gaussian processes thereon. For example, while the latter perform well on high-altitude wind data, they struggle with the more intricate, nonstationary patterns at low altitudes. Our models significantly improve performance in these settings, enhancing prediction quality and uncertainty calibration, and remain robust to overfitting, reverting to shallow models when additional complexity is unneeded. We further showcase our models on Bayesian optimisation problems on manifolds, using stylised examples motivated by robotics, and obtain substantial improvements in later stages of the optimisation process. Finally, we show our models to have potential for speeding up inference for non-manifold data, when, and if, it can be mapped to a proxy manifold well enough.