# GNet: A scalable and flexible Gaussian process network with nonparametric neurons

arXiv: https://arxiv.org/abs/2607.10735 · HTML: https://mlsanity.com/paper/2607.10735/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 63 references that are in the map.
- First posted on arXiv 12 Jul 2026.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [Deep Learning in Neural Networks: An Overview](https://mlsanity.com/paper/1404.7828/) (2014)
- [KAN: Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2404.19756/) (2024)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [FALKON: An Optimal Large Scale Kernel Method](https://mlsanity.com/paper/1705.10958/) (2017)
- [Hierarchical Nearest-Neighbor Gaussian Process Models for Large Geostatistical Datasets](https://mlsanity.com/paper/1406.7343/) (2014)
- [A general framework for Vecchia approximations of Gaussian processes](https://mlsanity.com/paper/1708.06302/) (2017)
- [Learning Activation Functions to Improve Deep Neural Networks](https://mlsanity.com/paper/1412.6830/) (2014)
- [Probabilistic Integration: A Role in Statistical Computation?](https://mlsanity.com/paper/1512.00933/) (2015)
- [Randomly Projected Additive Gaussian Processes for Regression](https://mlsanity.com/paper/1912.12834/) (2019)
- [Scaled Vecchia approximation for fast computer-model emulation](https://mlsanity.com/paper/2005.00386/) (2020)
- [State Space Gaussian Processes with Non-Gaussian Likelihood](https://mlsanity.com/paper/1802.04846/) (2018)
- [Neural network with optimal neuron activation functions based on additive Gaussian process regression](https://mlsanity.com/paper/2301.05567/) (2023)
- [Radial Neighbors for Provably Accurate Scalable Approximations of Gaussian Processes](https://mlsanity.com/paper/2211.14692/) (2022)
- [Gaussian Process Neurons Learn Stochastic Activation Functions](https://mlsanity.com/paper/1711.11059/) (2017)
- [All Emulators are Wrong, Many are Useful, and Some are More Useful Than Others: A Reproducible Comparison of Computer Model Surrogates](https://mlsanity.com/paper/2512.09060/) (2025)
- [The inverse Kalman filter](https://mlsanity.com/paper/2407.10089/) (2024)

## Abstract (arXiv)

We build GNet, a scalable and flexible Gaussian process network with nonparametric activation functions. The key computational contribution is to make estimation of GP activations computationally scalable without forming covariance matrices explicitly. We provide theoretical studies comparing function classes with a fixed architecture, and derive predictive risk bounds for GNets. Using essentially the same architecture and optimization settings, GNet shows competitive performance across a diverse range of test problems, including predicting nonlinear functions, nonparametric regression of real-world data, and predicting one-body direct correlation functions with high-dimensional inputs in classical density functional theory. The strong performance of GNet suggests that flexible nonparametric neurons can substantially reduce the number of trainable parameters while remaining computationally scalable.