# A Sieve Stochastic Gradient Descent Estimator for Online Nonparametric Regression in Sobolev ellipsoids

arXiv: https://arxiv.org/abs/2104.00846 · HTML: https://mlsanity.com/paper/2104.00846/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2021.

## Built on by (most cited first, 3 of 3)

- [Online Estimation with Rolling Validation: Adaptive Nonparametric Estimation with Streaming Data](https://mlsanity.com/paper/2310.12140/) (2023)
- [Stochastic Gradient Descent for Nonparametric Additive Regression](https://mlsanity.com/paper/2401.00691/) (2024)
- [Online Quantile Regression for Nonparametric Additive Models](https://mlsanity.com/paper/2604.08969/) (2026)

## Abstract (arXiv)

The goal of regression is to recover an unknown underlying function that best links a set of predictors to an outcome from noisy observations. In nonparametric regression, one assumes that the regression function belongs to a pre-specified infinite-dimensional function space (the hypothesis space). In the online setting, when the observations come in a stream, it is computationally-preferable to iteratively update an estimate rather than refitting an entire model repeatedly. Inspired by nonparametric sieve estimation and stochastic approximation methods, we propose a sieve stochastic gradient descent estimator (Sieve-SGD) when the hypothesis space is a Sobolev ellipsoid. We show that Sieve-SGD has rate-optimal mean squared error (MSE) under a set of simple and direct conditions. The proposed estimator can be constructed with a low computational (time and space) expense: We also formally show that Sieve-SGD requires almost minimal memory usage among all statistically rate-optimal estimators.