# Conditional Mean and Variance Estimation via textitk-NN Algorithm with Automated Variance Selection

arXiv: https://arxiv.org/abs/2402.01635 · HTML: https://mlsanity.com/paper/2402.01635/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 59 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Builds on (most cited first, 15 of 15)

- [Conformal prediction beyond exchangeability](https://mlsanity.com/paper/2202.13415/) (2022)
- [Deep learning: a statistical viewpoint](https://mlsanity.com/paper/2103.09177/) (2021)
- [FALKON: An Optimal Large Scale Kernel Method](https://mlsanity.com/paper/1705.10958/) (2017)
- [Approximate Nearest Neighbor Search in High Dimensions](https://mlsanity.com/paper/1806.09823/) (2018)
- [Survival regression with accelerated failure time model in XGBoost](https://mlsanity.com/paper/2006.04920/) (2020)
- [Efficient multivariate entropy estimation via k-nearest neighbour distances](https://mlsanity.com/paper/1606.00304/) (2016)
- [Sparse Regression: Scalable algorithms and empirical performance](https://mlsanity.com/paper/1902.06547/) (2019)
- [Universal consistency and rates of convergence of multiclass prototype algorithms in metric spaces](https://mlsanity.com/paper/2010.00636/) (2020)
- [Decorrelated Variable Importance](https://mlsanity.com/paper/2111.10853/) (2021)
- [Minimax Rate Optimal Adaptive Nearest Neighbor Classification and Regression](https://mlsanity.com/paper/1910.10513/) (2019)
- [A study of the classification of low-dimensional data with supervised manifold learning](https://mlsanity.com/paper/1507.05880/) (2015)
- [Nearest Neighbor and Kernel Survival Analysis: Nonasymptotic Error Bounds and Strong Consistency Rates](https://mlsanity.com/paper/1905.05285/) (2019)
- [Metric-valued regression](https://mlsanity.com/paper/2202.03045/) (2022)
- [Nearest neighbor empirical processes](https://mlsanity.com/paper/2110.15083/) (2021)
- [Variance estimation in graphs with the fused lasso](https://mlsanity.com/paper/2207.12638/) (2022)

## Built on by (most cited first, 2 of 2)

- [Fast Computation of Leave-One-Out Cross-Validation for k-NN Regression](https://mlsanity.com/paper/2405.04919/) (2024)
- [Model-Free Kernel Conformal Depth Measures Algorithm for Uncertainty Quantification in Regression Models in Separable Hilbert Spaces](https://mlsanity.com/paper/2506.08325/) (2025)

## Abstract (arXiv)

We introduce a novel textitk-nearest neighbor (textitk-NN) regression method for joint estimation of the conditional mean and variance. The proposed algorithm preserves the computational efficiency and manifold-learning capabilities of classical non-parametric textitk-NN models, while integrating a data-driven variable selection step that improves empirical performance. By accurately estimating both conditional mean and variance regression functions, the method effectively reconstructs the conditional distribution and density functions for multiple families of scale-and-localization generative models. We show that our estimator can achieve fast convergence rates, and we derive practical rules for selecting the smoothing parameter~k that enhance the precision of the algorithm in finite sample regimes. Extensive simulations for low, moderate and large-dimensional covariate spaces, together with a real-world biomedical application, demonstrate that the proposed method can consistently outperform the conventional textitk-NN regression algorithm while being more interpretable in the model output.