# Learning "best" kernels from data in Gaussian process regression. With application to aerodynamics

arXiv: https://arxiv.org/abs/2206.02563 · HTML: https://mlsanity.com/paper/2206.02563/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 105 references that are in the map.
- First posted on arXiv 3 Jun 2022.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Acceleration](https://mlsanity.com/paper/1809.11165/) (2018)
- [Solving and Learning Nonlinear PDEs with Gaussian Processes](https://mlsanity.com/paper/2103.12959/) (2021)
- [Numerical Gaussian Processes for Time-dependent and Non-linear Partial Differential Equations](https://mlsanity.com/paper/1703.10230/) (2017)
- [Operator-valued Kernels for Learning from Functional Response Data](https://mlsanity.com/paper/1510.08231/) (2015)
- [Kernel Flows: from learning kernels from data into the abyss](https://mlsanity.com/paper/1808.04475/) (2018)
- [Learning dynamical systems from data: a simple cross-validation perspective](https://mlsanity.com/paper/2007.05074/) (2020)
- [Polynomial-Chaos-based Kriging](https://mlsanity.com/paper/1502.03939/) (2015)
- [Consistency of Empirical Bayes And Kernel Flow For Hierarchical Parameter Estimation](https://mlsanity.com/paper/2005.11375/) (2020)
- [Data-driven geophysical forecasting: Simple, low-cost, and accurate baselines with kernel methods](https://mlsanity.com/paper/2103.10935/) (2021)
- [Deep regularization and direct training of the inner layers of Neural Networks with Kernel Flows](https://mlsanity.com/paper/2002.08335/) (2020)

## Built on by (most cited first, 9 of 9)

- [Learning Dynamical Systems from Data: A Simple Cross-Validation Perspective, Part V: Sparse Kernel Flows for 132 Chaotic Dynamical Systems](https://mlsanity.com/paper/2301.10321/) (2023)
- [Gaussian Process Hydrodynamics](https://mlsanity.com/paper/2209.10707/) (2022)
- [Bridging Algorithmic Information Theory and Machine Learning: A New Approach to Kernel Learning](https://mlsanity.com/paper/2311.12624/) (2023)
- [Learning Collective Behaviors from Observation](https://mlsanity.com/paper/2311.00875/) (2023)
- [Data-Driven Model Selections of Second-Order Particle Dynamics via Integrating Gaussian Processes with Low-Dimensional Interacting Structures](https://mlsanity.com/paper/2311.00902/) (2023)
- [Physics-informed, boundary-constrained Gaussian process regression for the reconstruction of fluid flow fields](https://mlsanity.com/paper/2507.17582/) (2025)
- [Decoupled-Value Attention for Prior-Data Fitted Networks: GP Inference for Physical Equations](https://mlsanity.com/paper/2509.20950/) (2025)
- [Data-driven Learning of Interaction Laws in Multispecies Particle Systems with Gaussian Processes: Convergence Theory and Applications](https://mlsanity.com/paper/2511.02053/) (2025)
- [Supervised Learning of Random Neural Architectures Structured by Latent Random Fields on Compact Boundaryless Multiply-Connected Manifolds](https://mlsanity.com/paper/2512.10407/) (2025)

## Abstract (arXiv)

This paper introduces algorithms to select/design kernels in Gaussian process regression/kriging surrogate modeling techniques. We adopt the setting of kernel method solutions in ad hoc functional spaces, namely Reproducing Kernel Hilbert Spaces (RKHS), to solve the problem of approximating a regular target function given observations of it, i.e. supervised learning. A first class of algorithms is kernel flow, which was introduced in the context of classification in machine learning. It can be seen as a cross-validation procedure whereby a "best" kernel is selected such that the loss of accuracy incurred by removing some part of the dataset (typically half of it) is minimized. A second class of algorithms is called spectral kernel ridge regression, and aims at selecting a "best" kernel such that the norm of the function to be approximated is minimal in the associated RKHS. Within Mercer's theorem framework, we obtain an explicit construction of that "best" kernel in terms of the main features of the target function. Both approaches of learning kernels from data are illustrated by numerical examples on synthetic test functions, and on a classical test case in turbulence modeling validation for transonic flows about a two-dimensional airfoil.