# Kernel Mean Shrinkage Estimators

arXiv: https://arxiv.org/abs/1405.5505 · HTML: https://mlsanity.com/paper/1405.5505/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2014.

## Built on by (most cited first, 16 of 16)

- [Deconditional Downscaling with Gaussian Processes](https://mlsanity.com/paper/2105.12909/) (2021)
- [Conditional Bayesian Quadrature](https://mlsanity.com/paper/2406.16530/) (2024)
- [A Dictionary of Closed-Form Kernel Mean Embeddings](https://mlsanity.com/paper/2504.18830/) (2025)
- [Variance-Aware Estimation of Kernel Mean Embedding](https://mlsanity.com/paper/2210.06672/) (2022)
- [Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigation](https://mlsanity.com/paper/2510.08962/) (2025)
- [Kernel Relative-prototype Spectral Filtering for Few-shot Learning](https://mlsanity.com/paper/2207.11685/) (2022)
- [Variance Matters: Improving Domain Adaptation via Stratified Sampling](https://mlsanity.com/paper/2512.05226/) (2025)
- [Toward Fair Facial Expression Recognition with Improved Distribution Alignment](https://mlsanity.com/paper/2306.06696/) (2023)
- [A Survey of Learning on Small Data: Generalization, Optimization, and Challenge](https://mlsanity.com/paper/2207.14443/) (2022)
- [Kernel Mean Estimation by Marginalized Corrupted Distributions](https://mlsanity.com/paper/2107.04855/) (2021)
- [Non-Linear Spectral Dimensionality Reduction Under Uncertainty](https://mlsanity.com/paper/2202.04678/) (2022)
- [Shrinkage Estimation of Higher Order Bochner Integrals](https://mlsanity.com/paper/2207.06357/) (2022)
- [Returning The Favour: When Regression Benefits From Probabilistic Causal Knowledge](https://mlsanity.com/paper/2301.11214/) (2023)
- [SENet: A Spectral Filtering Approach to Represent Exemplars for Few-shot Learning](https://mlsanity.com/paper/2305.18970/) (2023)
- [Entrywise error bounds for low-rank approximations of kernel matrices](https://mlsanity.com/paper/2405.14494/) (2024)
- [On Kernel Eigen-alignments of KRR: Reconstruction and Generalization](https://mlsanity.com/paper/2605.15240/) (2026)

## Abstract (arXiv)

A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in RKHSs. Given a finite sample, an empirical average has been used commonly as a standard estimator of the true kernel mean. Despite a widespread use of this estimator, we show that it can be improved thanks to the well-known Stein phenomenon. We propose a new family of estimators called kernel mean shrinkage estimators (KMSEs), which benefit from both theoretical justifications and good empirical performance. The results demonstrate that the proposed estimators outperform the standard one, especially in a "large d, small n" paradigm.