# Optimal Sampling Density for Nonparametric Regression

arXiv: https://arxiv.org/abs/2105.11990 · HTML: https://mlsanity.com/paper/2105.11990/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 62 references that are in the map.
- First posted on arXiv 25 May 2021.

## Builds on (most cited first, 6 of 6)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Unmasking Clever Hans Predictors and Assessing What Machines Really Learn](https://mlsanity.com/paper/1902.10178/) (2019)
- [Pool-Based Sequential Active Learning for Regression](https://mlsanity.com/paper/1805.04735/) (2018)
- [Pool-Based Unsupervised Active Learning for Regression Using Iterative Representativeness-Diversity Maximization (iRDM)](https://mlsanity.com/paper/2003.07658/) (2020)
- [Prediction of Atomization Energy Using Graph Kernel and Active Learning](https://mlsanity.com/paper/1810.07310/) (2018)
- [Active Learning Approach to Optimization of Experimental Control](https://mlsanity.com/paper/2003.11804/) (2020)

## Abstract (arXiv)

We propose a novel active learning strategy for regression, which is model-agnostic, robust against model mismatch, and interpretable. Assuming that a small number of initial samples are available, we derive the optimal training density that minimizes the generalization error of local polynomial smoothing (LPS) with its kernel bandwidth tuned locally: We adopt the mean integrated squared error (MISE) as a generalization criterion, and use the asymptotic behavior of the MISE as well as the locally optimal bandwidths (LOB) - the bandwidth function that minimizes MISE in the asymptotic limit. The asymptotic expression of our objective then reveals the dependence of the MISE on the training density, enabling analytic minimization. As a result,we obtain the optimal training density in a closed-form. The almost model-free nature of our approach thus helps to encode the essential properties of the target problem, providing a robust and model-agnostic active learning strategy. Furthermore, the obtained training density factorizes the influence of local function complexity, noise level and test density in a transparent and interpretable way. We validate our theory in numerical simulations, and show that the proposed active learning method outperforms the existing state-of-the-art model-agnostic approaches.