# Distributional encoding for Gaussian process regression with qualitative inputs

arXiv: https://arxiv.org/abs/2506.04813 · HTML: https://mlsanity.com/paper/2506.04813/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 12 of 12)

- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Statistical and Topological Properties of Sliced Probability Divergences](https://mlsanity.com/paper/2003.05783/) (2020)
- [Statistical, Robustness, and Computational Guarantees for Sliced Wasserstein Distances](https://mlsanity.com/paper/2210.09160/) (2022)
- [A Latent Variable Approach to Gaussian Process Modeling with Qualitative and Quantitative Factors](https://mlsanity.com/paper/1806.07504/) (2018)
- [Group kernels for Gaussian process metamodels with categorical inputs](https://mlsanity.com/paper/1802.02368/) (2018)
- [Latent Map Gaussian Processes for Mixed Variable Metamodeling](https://mlsanity.com/paper/2102.03935/) (2021)
- [A mixed-categorical correlation kernel for Gaussian process](https://mlsanity.com/paper/2211.08262/) (2022)
- [Distribution Regression with Sliced Wasserstein Kernels](https://mlsanity.com/paper/2202.03926/) (2022)
- [A Gaussian Process Regression Model for Distribution Inputs](https://mlsanity.com/paper/1701.09055/) (2017)
- [Sliced-Wasserstein Estimation with Spherical Harmonics as Control Variates](https://mlsanity.com/paper/2402.01493/) (2024)
- [Fast calculation of Gaussian Process multiple-fold cross-validation residuals and their covariances](https://mlsanity.com/paper/2101.03108/) (2021)
- [Gaussian Process Models with Low-Rank Correlation Matrices for Both Continuous and Categorical Inputs](https://mlsanity.com/paper/2010.02574/) (2020)

## Built on by (most cited first, 1 of 1)

- [A reproducible comparative study of categorical kernels for Gaussian process regression, with new clustering-based nested kernels](https://mlsanity.com/paper/2510.01840/) (2025)

## Abstract (arXiv)

Gaussian Process (GP) regression is a popular and sample-efficient approach for many engineering applications, where observations are expensive to acquire, and is also a central ingredient of Bayesian optimization (BO), a highly prevailing method for the optimization of black-box functions. However, when all or some input variables are categorical, building a predictive and computationally efficient GP remains challenging. Starting from the naive target encoding idea, where the original categorical values are replaced with the mean of the target variable for that category, we propose a generalization based on distributional encoding (DE) which makes use of all samples of the target variable for a category. To handle this type of encoding inside the GP, we build upon recent results on characteristic kernels for probability distributions, based on the maximum mean discrepancy and the Wasserstein distance. We also discuss several extensions for classification, multi-task learning and incorporation or auxiliary information. Our approach is validated empirically, and we demonstrate state-of-the-art predictive performance on a variety of synthetic and real-world datasets. DE is naturally complementary to recent advances in BO over discrete and mixed-spaces.