# Data eccentricity, asymptotics of Gaussian RBF reproducing kernel Hilbert space, and kernel PCA

arXiv: https://arxiv.org/abs/2607.21823 · HTML: https://mlsanity.com/paper/2607.21823/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 63 references that are in the map.
- First posted on arXiv 23 Jul 2026.

## Builds on (most cited first, 13 of 13)

- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth](https://mlsanity.com/paper/2010.15327/) (2020)
- [Emerging Cross-lingual Structure in Pretrained Language Models](https://mlsanity.com/paper/1911.01464/) (2019)
- [Entangled Watermarks as a Defense against Model Extraction](https://mlsanity.com/paper/2002.12200/) (2020)
- [Random Fourier Features for Kernel Ridge Regression: Approximation Bounds and Statistical Guarantees](https://mlsanity.com/paper/1804.09893/) (2018)
- [Universality and individuality in neural dynamics across large populations of recurrent networks](https://mlsanity.com/paper/1907.08549/) (2019)
- [Graph Multiview Canonical Correlation Analysis](https://mlsanity.com/paper/1811.12345/) (2018)
- [Streaming Kernel Principal Component Analysis](https://mlsanity.com/paper/1512.05059/) (2015)
- [Pyrcca: regularized kernel canonical correlation analysis in Python and its applications to neuroimaging](https://mlsanity.com/paper/1503.01538/) (2015)
- [Convex Formulation for Kernel PCA and its Use in Semi-Supervised Learning](https://mlsanity.com/paper/1610.06811/) (2016)
- [Conformalized Kernel Ridge Regression](https://mlsanity.com/paper/1609.05959/) (2016)
- [Gaussian RBF Centered Kernel Alignment (CKA) in the Large Bandwidth Limit](https://mlsanity.com/paper/2112.09305/) (2021)

## Abstract (arXiv)

We show that, up to isotropic scaling, the Gaussian RBF reproducing kernel Hilbert space (RKHS) is asymptotically isometric to Euclidean space in the large bandwidth limit. This strongly suggests that kernel-based constructions reliant on metric properties of the RKHS will yield results for Gaussian RBF kernels that similarly approach those of linear kernels for large bandwidths. The asymptotic behavior of Gaussian CKA can be understood in this light. We further consider kernel PCA, showing that Gaussian RBF eigenvalues, eigenprojections, and principal components all converge to those of classical (linear) PCA as bandwidth σ rightarrow ∞. For a given data representation, both the RKHS feature embeddings and the orthogonal PCA eigenframes of the two kernel types differ asymptotically by a geometric similarity transformation, up to a residual of size O left (fracρσ right )2, where ρ is a measure of geometric eccentricity of the representation, equal to the ratio of maximum to median pairwise distance between data examples. Experiments over a diverse collection of data sets demonstrate that ρ provides a simple and reliable predictor of dataset-specific convergence behavior in the top principal directions.