# A Spectral Series Approach to High-Dimensional Nonparametric Regression

arXiv: https://arxiv.org/abs/1602.00355 · HTML: https://mlsanity.com/paper/1602.00355/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Feb 2016.

## Built on by (most cited first, 8 of 8)

- [Minimax Optimal Regression over Sobolev Spaces via Laplacian Regularization on Neighborhood Graphs](https://mlsanity.com/paper/2106.01529/) (2021)
- [Minimax Optimal Regression over Sobolev Spaces via Laplacian Eigenmaps on Neighborhood Graphs](https://mlsanity.com/paper/2111.07394/) (2021)
- [Adaptive and non-adaptive minimax rates for weighted Laplacian-eigenmap based nonparametric regression](https://mlsanity.com/paper/2311.00140/) (2023)
- [Multiscale regression on unknown manifolds](https://mlsanity.com/paper/2101.05119/) (2021)
- [Flexible conditional density estimation for time series](https://mlsanity.com/paper/2301.09671/) (2023)
- [Skeleton Regression: A Graph-Based Approach to Estimation with Manifold Structure](https://mlsanity.com/paper/2303.11786/) (2023)
- [Expertise-based Weighting for Regression Models with Noisy Labels](https://mlsanity.com/paper/2305.07430/) (2023)
- [Trustworthy Predictive Distributions for Tail Events with Semiparametric Diagnostic Transport Maps](https://mlsanity.com/paper/2603.11229/) (2026)

## Abstract (arXiv)

A key question in modern statistics is how to make fast and reliable inferences for complex, high-dimensional data. While there has been much interest in sparse techniques, current methods do not generalize well to data with nonlinear structure. In this work, we present an orthogonal series estimator for predictors that are complex aggregate objects, such as natural images, galaxy spectra, trajectories, and movies. Our series approach ties together ideas from kernel machine learning, and Fourier methods. We expand the unknown regression on the data in terms of the eigenfunctions of a kernel-based operator, and we take advantage of orthogonality of the basis with respect to the underlying data distribution, P, to speed up computations and tuning of parameters. If the kernel is appropriately chosen, then the eigenfunctions adapt to the intrinsic geometry and dimension of the data. We provide theoretical guarantees for a radial kernel with varying bandwidth, and we relate smoothness of the regression function with respect to P to sparsity in the eigenbasis. Finally, using simulated and real-world data, we systematically compare the performance of the spectral series approach with classical kernel smoothing, k-nearest neighbors regression, kernel ridge regression, and state-of-the-art manifold and local regression methods.