# An Online Projection Estimator for Nonparametric Regression in Reproducing Kernel Hilbert Spaces

arXiv: https://arxiv.org/abs/2104.00780 · HTML: https://mlsanity.com/paper/2104.00780/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Apr 2021.

## Built on by (most cited first, 2 of 2)

- [Statistical inference using Regularized M-estimation in the reproducing kernel Hilbert space for handling missing data](https://mlsanity.com/paper/2107.07371/) (2021)
- [Online Estimation with Rolling Validation: Adaptive Nonparametric Estimation with Streaming Data](https://mlsanity.com/paper/2310.12140/) (2023)

## Abstract (arXiv)

The goal of nonparametric regression is to recover an underlying regression function from noisy observations, under the assumption that the regression function belongs to a pre-specified infinite dimensional function space. In the online setting, when the observations come in a stream, it is generally computationally infeasible to refit the whole model repeatedly. There are as of yet no methods that are both computationally efficient and statistically rate-optimal. In this paper, we propose an estimator for online nonparametric regression. Notably, our estimator is an empirical risk minimizer (ERM) in a deterministic linear space, which is quite different from existing methods using random features and functional stochastic gradient. Our theoretical analysis shows that this estimator obtains rate-optimal generalization error when the regression function is known to live in a reproducing kernel Hilbert space. We also show, theoretically and empirically, that the computational expense of our estimator is much lower than other rate-optimal estimators proposed for this online setting.