# On the optimization of hyperparameters in Gaussian process regression with the help of low-order high-dimensional model representation

arXiv: https://arxiv.org/abs/2112.01374 · HTML: https://mlsanity.com/paper/2112.01374/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Nov 2021.

## Built on by (most cited first, 6 of 6)

- [Exact Gaussian Processes for Massive Datasets via Non-Stationary Sparsity-Discovering Kernels](https://mlsanity.com/paper/2205.09070/) (2022)
- [Neural network with optimal neuron activation functions based on additive Gaussian process regression](https://mlsanity.com/paper/2301.05567/) (2023)
- [The loss of the property of locality of the kernel in high-dimensional Gaussian process regression on the example of the fitting of molecular potential energy surfaces](https://mlsanity.com/paper/2211.11170/) (2022)
- [Orders-of-coupling representation with a single neural network with optimal neuron activation functions and without nonlinear parameter optimization](https://mlsanity.com/paper/2302.12013/) (2023)
- [A kinetic-based regularization method for data science applications](https://mlsanity.com/paper/2503.04857/) (2025)
- [Gaussian Process Regression -- Neural Network Hybrid with Optimized Redundant Coordinates](https://mlsanity.com/paper/2509.08457/) (2025)

## Abstract (arXiv)

When the data are sparse, optimization of hyperparameters of the kernel in Gaussian process regression by the commonly used maximum likelihood estimation (MLE) criterion often leads to overfitting. We show that choosing hyperparameters (in this case, kernel length parameter and regularization parameter) based on a criterion of the completeness of the basis in the corresponding linear regression problem is superior to MLE. We show that this is facilitated by the use of high-dimensional model representation (HDMR) whereby a low-order HDMR representation can provide reliable reference functions and large synthetic test data sets needed for basis parameter optimization even when the original data are few.