# Consistent Interpolating Ensembles via the Manifold-Hilbert Kernel

arXiv: https://arxiv.org/abs/2205.09342 · HTML: https://mlsanity.com/paper/2205.09342/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 47 references that are in the map.
- First posted on arXiv 19 May 2022.

## Builds on (most cited first, 12 of 12)

- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [A Random Forest Guided Tour](https://mlsanity.com/paper/1511.05741/) (2015)
- [Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolate](https://mlsanity.com/paper/1806.05161/) (2018)
- [Does data interpolation contradict statistical optimality?](https://mlsanity.com/paper/1806.09471/) (2018)
- [Consistency of random forests](https://mlsanity.com/paper/1405.2881/) (2014)
- [Explaining the Success of AdaBoost and Random Forests as Interpolating Classifiers](https://mlsanity.com/paper/1504.07676/) (2015)
- [Persistence Fisher Kernel: A Riemannian Manifold Kernel for Persistence Diagrams](https://mlsanity.com/paper/1802.03569/) (2018)
- [Kernel Methods on Riemannian Manifolds with Gaussian RBF Kernels](https://mlsanity.com/paper/1412.0265/) (2014)
- [The Random Forest Kernel and other kernels for big data from random partitions](https://mlsanity.com/paper/1402.4293/) (2014)
- [A Riemannian Framework for Statistical Analysis of Topological Persistence Diagrams](https://mlsanity.com/paper/1605.08912/) (2016)
- [Principal Boundary on Riemannian Manifolds](https://mlsanity.com/paper/1711.06705/) (2017)

## Built on by (most cited first, 1 of 1)

- [Mind the spikes: Benign overfitting of kernels and neural networks in fixed dimension](https://mlsanity.com/paper/2305.14077/) (2023)

## Abstract (arXiv)

Recent research in the theory of overparametrized learning has sought to establish generalization guarantees in the interpolating regime. Such results have been established for a few common classes of methods, but so far not for ensemble methods. We devise an ensemble classification method that simultaneously interpolates the training data, and is consistent for a broad class of data distributions. To this end, we define the manifold-Hilbert kernel for data distributed on a Riemannian manifold. We prove that kernel smoothing regression using the manifold-Hilbert kernel is weakly consistent in the setting of Devroye et al. 1998. For the sphere, we show that the manifold-Hilbert kernel can be realized as a weighted random partition kernel, which arises as an infinite ensemble of partition-based classifiers.