# Nonparametric Score Estimators

arXiv: https://arxiv.org/abs/2005.10099 · HTML: https://mlsanity.com/paper/2005.10099/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 May 2020.

## Built on by (most cited first, 23 of 23)

- [Estimating High Order Gradients of the Data Distribution by Denoising](https://mlsanity.com/paper/2111.04726/) (2021)
- [Optimal score estimation via empirical Bayes smoothing](https://mlsanity.com/paper/2402.07747/) (2024)
- [Nonparametric Generative Modeling with Conditional Sliced-Wasserstein Flows](https://mlsanity.com/paper/2305.02164/) (2023)
- [Linear Causal Representation Learning from Unknown Multi-node Interventions](https://mlsanity.com/paper/2406.05937/) (2024)
- [Differentiable Neural Networks with RePU Activation: with Applications to Score Estimation and Isotonic Regression](https://mlsanity.com/paper/2305.00608/) (2023)
- [Bridging the Sim-to-Real Gap with Bayesian Inference](https://mlsanity.com/paper/2403.16644/) (2024)
- [Algorithmic Differentiation for Automated Modeling of Machine Learned Force Fields](https://mlsanity.com/paper/2208.12104/) (2022)
- [A Skewness-Based Criterion for Addressing Heteroscedastic Noise in Causal Discovery](https://mlsanity.com/paper/2410.06407/) (2024)
- [Distinguishing Cause from Effect with Causal Velocity Models](https://mlsanity.com/paper/2502.05122/) (2025)
- [MARS: Meta-Learning as Score Matching in the Function Space](https://mlsanity.com/paper/2210.13319/) (2022)
- [Active Slices for Sliced Stein Discrepancy](https://mlsanity.com/paper/2102.03159/) (2021)
- [AutoTransfer: Subject Transfer Learning with Censored Representations on Biosignals Data](https://mlsanity.com/paper/2112.09796/) (2021)
- [Nonparametric Automatic Differentiation Variational Inference with Spline Approximation](https://mlsanity.com/paper/2403.06302/) (2024)
- [Aligning Few-Step Generative Models by Amortizing Sample-based Variational Inference](https://mlsanity.com/paper/2605.26552/) (2026)
- [Constraint-Aware Flow Matching via Randomized Exploration](https://mlsanity.com/paper/2508.13316/) (2025)
- [A Kernelised Stein Statistic for Assessing Implicit Generative Models](https://mlsanity.com/paper/2206.00149/) (2022)
- [Nonlinear Multiple Response Regression and Learning of Latent Spaces](https://mlsanity.com/paper/2503.21608/) (2025)
- [Multi-Source Transfer Learning of Sparse Single-Index Models](https://mlsanity.com/paper/2606.29658/) (2026)
- [Nonparametric learning of heterogeneous graphical model on network-linked data](https://mlsanity.com/paper/2507.01473/) (2025)
- [Simulation Priors for Data-Efficient Deep Learning](https://mlsanity.com/paper/2509.05732/) (2025)
- [Variance-Reduced Diffusion Sampling via Target Score Identity](https://mlsanity.com/paper/2601.01594/) (2026)
- [Non-parametric recovery of causal diffusion mechanisms from steady-state observations](https://mlsanity.com/paper/2606.30467/) (2026)
- [Local Fisher Information Enables Sparse Causal Discovery](https://mlsanity.com/paper/2610.04291/) (2026)

## Abstract (arXiv)

Estimating the score, i.e., the gradient of log density function, from a set of samples generated by an unknown distribution is a fundamental task in inference and learning of probabilistic models that involve flexible yet intractable densities. Kernel estimators based on Stein's methods or score matching have shown promise, however their theoretical properties and relationships have not been fully-understood. We provide a unifying view of these estimators under the framework of regularized nonparametric regression. It allows us to analyse existing estimators and construct new ones with desirable properties by choosing different hypothesis spaces and regularizers. A unified convergence analysis is provided for such estimators. Finally, we propose score estimators based on iterative regularization that enjoy computational benefits from curl-free kernels and fast convergence.