# Non-Stationary Gaussian Process Regression with Hamiltonian Monte Carlo

arXiv: https://arxiv.org/abs/1508.04319 · HTML: https://mlsanity.com/paper/1508.04319/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Aug 2015.

## Built on by (most cited first, 29 of 29)

- [Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial](https://mlsanity.com/paper/2305.04933/) (2023)
- [Kernel Neural Operators (KNOs) for Scalable, Memory-efficient, Geometrically-flexible Operator Learning](https://mlsanity.com/paper/2407.00809/) (2024)
- [Partitioned Active Learning for Heterogeneous Systems](https://mlsanity.com/paper/2105.08547/) (2021)
- [Efficient Transformed Gaussian Processes for Non-Stationary Dependent Multi-class Classification](https://mlsanity.com/paper/2205.15008/) (2022)
- [Active learning-assisted neutron spectroscopy with log-Gaussian processes](https://mlsanity.com/paper/2209.00980/) (2022)
- [Hierarchical-Hyperplane Kernels for Actively Learning Gaussian Process Models of Nonstationary Systems](https://mlsanity.com/paper/2303.10022/) (2023)
- [Mixtures of Gaussian Process Experts with SMC2](https://mlsanity.com/paper/2208.12830/) (2022)
- [Kernel Learning for Explainable Climate Science](https://mlsanity.com/paper/2209.04947/) (2022)
- [The Normalized Cross Density Functional: A Framework to Quantify Statistical Dependence for Random Processes](https://mlsanity.com/paper/2212.04631/) (2022)
- [Thin and Deep Gaussian Processes](https://mlsanity.com/paper/2310.11527/) (2023)
- [Accurate Remaining Useful Life Prediction with Uncertainty Quantification: a Deep Learning and Nonstationary Gaussian Process Approach](https://mlsanity.com/paper/2109.12111/) (2021)
- [Bayesian Optimization with Informative Covariance](https://mlsanity.com/paper/2208.02704/) (2022)
- [Uncertainty Disentanglement with Non-stationary Heteroscedastic Gaussian Processes for Active Learning](https://mlsanity.com/paper/2210.10964/) (2022)
- [Nonstationary Sparse Spectral Permanental Process](https://mlsanity.com/paper/2410.03581/) (2024)
- [Deep Jump Gaussian Processes for Surrogate Modeling of High-Dimensional Piecewise Continuous Functions](https://mlsanity.com/paper/2510.21974/) (2025)
- [Hierarchical Non-Stationary Temporal Gaussian Processes With L1-Regularization](https://mlsanity.com/paper/2105.09695/) (2021)
- [Learning non-stationary and discontinuous functions using clustering, classification and Gaussian process modelling](https://mlsanity.com/paper/2211.16909/) (2022)
- [Polynomial Chaos Expanded Gaussian Process](https://mlsanity.com/paper/2405.01052/) (2024)
- [Non-stationary and Sparsely-correlated Multi-output Gaussian Process with Spike-and-Slab Prior](https://mlsanity.com/paper/2409.03149/) (2024)
- [Mind the Jumps: A Scalable Robust Local Gaussian Process for Multidimensional Response Surfaces with Discontinuities](https://mlsanity.com/paper/2512.12574/) (2025)
- [Correcting Boundary Bias and Observation Independence in Bayesian Experimental Design](https://mlsanity.com/paper/2602.01898/) (2026)
- [A New Reliable & Parsimonious Learning Strategy Comprising Two Layers of Gaussian Processes, to Address Inhomogeneous Empirical Correlation Structures](https://mlsanity.com/paper/2404.12478/) (2024)
- [Theoretical Analysis of Heteroscedastic Gaussian Processes with Posterior Distributions](https://mlsanity.com/paper/2409.12622/) (2024)
- [Scalable Batch Bayesian Optimization Via Subspace Acquisition Functions](https://mlsanity.com/paper/2411.16206/) (2024)
- [SEEK: Self-adaptive Explainable Kernel For Nonstationary Gaussian Processes](https://mlsanity.com/paper/2503.14785/) (2025)
- [Improving Random Forests by Smoothing](https://mlsanity.com/paper/2505.06852/) (2025)
- [A Framework for Nonstationary Gaussian Processes with Neural Network Parameters](https://mlsanity.com/paper/2507.12262/) (2025)
- [Predicting Covariate-Driven Spatial Deformation for Nonstationary Gaussian Processes](https://mlsanity.com/paper/2604.27280/) (2026)
- [Covariance-Boosted Gaussian Processes for Spatiotemporal Irregularities](https://mlsanity.com/paper/2607.23018/) (2026)

## Abstract (arXiv)

We present a novel approach for fully non-stationary Gaussian process regression (GPR), where all three key parameters -- noise variance, signal variance and lengthscale -- can be simultaneously input-dependent. We develop gradient-based inference methods to learn the unknown function and the non-stationary model parameters, without requiring any model approximations. We propose to infer full parameter posterior with Hamiltonian Monte Carlo (HMC), which conveniently extends the analytical gradient-based GPR learning by guiding the sampling with model gradients. We also learn the MAP solution from the posterior by gradient ascent. In experiments on several synthetic datasets and in modelling of temporal gene expression, the nonstationary GPR is shown to be necessary for modeling realistic input-dependent dynamics, while it performs comparably to conventional stationary or previous non-stationary GPR models otherwise.