# Structural Kernel Search via Bayesian Optimization and Symbolical Optimal Transport

arXiv: https://arxiv.org/abs/2210.11836 · HTML: https://mlsanity.com/paper/2210.11836/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 21 Oct 2022.

## Builds on (most cited first, 11 of 11)

- [Deep Kernel Learning](https://mlsanity.com/paper/1511.02222/) (2015)
- [Neural Architecture Search with Bayesian Optimisation and Optimal Transport](https://mlsanity.com/paper/1802.07191/) (2018)
- [GPflow: A Gaussian process library using TensorFlow](https://mlsanity.com/paper/1610.08733/) (2016)
- [Gaussian Process Kernels for Pattern Discovery and Extrapolation](https://mlsanity.com/paper/1302.4245/) (2013)
- [Structure Discovery in Nonparametric Regression through Compositional Kernel Search](https://mlsanity.com/paper/1302.4922/) (2013)
- [The Promises and Pitfalls of Deep Kernel Learning](https://mlsanity.com/paper/2102.12108/) (2021)
- [ChemBO: Bayesian Optimization of Small Organic Molecules with Synthesizable Recommendations](https://mlsanity.com/paper/1908.01425/) (2019)
- [Automatic Construction and Natural-Language Description of Nonparametric Regression Models](https://mlsanity.com/paper/1402.4304/) (2014)
- [Optimal Transport Kernels for Sequential and Parallel Neural Architecture Search](https://mlsanity.com/paper/2006.07593/) (2020)
- [Differentiable Compositional Kernel Learning for Gaussian Processes](https://mlsanity.com/paper/1806.04326/) (2018)
- [Function-Space Distributions over Kernels](https://mlsanity.com/paper/1910.13565/) (2019)

## Built on by (most cited first, 8 of 8)

- [Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs](https://mlsanity.com/paper/2509.17998/) (2025)
- [Amortized Inference for Gaussian Process Hyperparameters of Structured Kernels](https://mlsanity.com/paper/2306.09819/) (2023)
- [Hierarchical-Hyperplane Kernels for Actively Learning Gaussian Process Models of Nonstationary Systems](https://mlsanity.com/paper/2303.10022/) (2023)
- [Automated Model Discovery via Multi-modal & Multi-step Pipeline](https://mlsanity.com/paper/2509.25946/) (2025)
- [Efficiently Computable Safety Bounds for Gaussian Processes in Active Learning](https://mlsanity.com/paper/2402.18260/) (2024)
- [Global Safe Sequential Learning via Efficient Knowledge Transfer](https://mlsanity.com/paper/2402.14402/) (2024)
- [Future Aware Safe Active Learning of Time Varying Systems using Gaussian Processes](https://mlsanity.com/paper/2405.10581/) (2024)
- [Batch Active Learning in Gaussian Process Regression using Derivatives](https://mlsanity.com/paper/2408.01861/) (2024)

## Abstract (arXiv)

Despite recent advances in automated machine learning, model selection is still a complex and computationally intensive process. For Gaussian processes (GPs), selecting the kernel is a crucial task, often done manually by the expert. Additionally, evaluating the model selection criteria for Gaussian processes typically scales cubically in the sample size, rendering kernel search particularly computationally expensive. We propose a novel, efficient search method through a general, structured kernel space. Previous methods solved this task via Bayesian optimization and relied on measuring the distance between GP's directly in function space to construct a kernel-kernel. We present an alternative approach by defining a kernel-kernel over the symbolic representation of the statistical hypothesis that is associated with a kernel. We empirically show that this leads to a computationally more efficient way of searching through a discrete kernel space.