# Gaussian Process Regression -- Neural Network Hybrid with Optimized Redundant Coordinates

arXiv: https://arxiv.org/abs/2509.08457 · HTML: https://mlsanity.com/paper/2509.08457/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 64 references that are in the map.
- First posted on arXiv 10 Sep 2025.

## Builds on (most cited first, 10 of 10)

- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Interpretable Scientific Discovery with Symbolic Regression: A Review](https://mlsanity.com/paper/2211.10873/) (2022)
- [A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledge](https://mlsanity.com/paper/2209.06257/) (2022)
- [Toward Physically Plausible Data-Driven Models: A Novel Neural Network Approach to Symbolic Regression](https://mlsanity.com/paper/2302.00773/) (2023)
- [On the optimization of hyperparameters in Gaussian process regression with the help of low-order high-dimensional model representation](https://mlsanity.com/paper/2112.01374/) (2021)
- [Random Sampling High Dimensional Model Representation Gaussian Process Regression (RS-HDMR-GPR) for representing multidimensional functions with machine-learned lower-dimensional terms allowing insight with a general method](https://mlsanity.com/paper/2012.02704/) (2020)
- [Neural network with optimal neuron activation functions based on additive Gaussian process regression](https://mlsanity.com/paper/2301.05567/) (2023)
- [Degeneration of kernel regression with Matern kernels into low-order polynomial regression in high dimension](https://mlsanity.com/paper/2311.10790/) (2023)
- [The loss of the property of locality of the kernel in high-dimensional Gaussian process regression on the example of the fitting of molecular potential energy surfaces](https://mlsanity.com/paper/2211.11170/) (2022)
- [Machine learning-guided construction of an analytic kinetic energy functional for orbital free density functional theory](https://mlsanity.com/paper/2502.05411/) (2025)

## Abstract (arXiv)

Recently, a Gaussian Process Regression - neural network (GPRNN) hybrid machine learning method was proposed, which is based on additive-kernel GPR in redundant coordinates constructed by rules [J. Phys. Chem. A 127 (2023) 7823]. The method combined the expressive power of an NN with the robustness of linear regression, in particular, with respect to overfitting when the number of neurons is increased beyond optimal. We introduce opt-GPRNN, in which the redundant coordinates of GPRNN are optimized with a Monte Carlo algorithm and show that when combined with optimization of redundant coordinates, GPRNN attains the lowest test set error with much fewer terms / neurons and retains the advantage of avoiding overfitting when the number of neurons is increased beyond optimal value. The method, opt-GPRNN possesses an expressive power closer to that of a multilayer NN and could obviate the need for deep NNs in some applications. With optimized redundant coordinates, a dimensionality reduction regime is also possible. Examples of application to machine learning an interatomic potential and materials informatics are given.