# Denoising Score Matching with Random Fourier Features

arXiv: https://arxiv.org/abs/2101.05239 · HTML: https://mlsanity.com/paper/2101.05239/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 37 references that are in the map.
- First posted on arXiv 13 Jan 2021.

## Builds on (most cited first, 19 of 19)

- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [An Introduction to Matrix Concentration Inequalities](https://mlsanity.com/paper/1501.01571/) (2015)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Stabilizing Training of Generative Adversarial Networks through Regularization](https://mlsanity.com/paper/1705.09367/) (2017)
- [Regularization by Denoising: Clarifications and New Interpretations](https://mlsanity.com/paper/1806.02296/) (2018)
- [Orthogonal Random Features](https://mlsanity.com/paper/1610.09072/) (2016)
- [The Unreasonable Effectiveness of Structured Random Orthogonal Embeddings](https://mlsanity.com/paper/1703.00864/) (2017)
- [Density Estimation in Infinite Dimensional Exponential Families](https://mlsanity.com/paper/1312.3516/) (2013)
- [A Linear-Time Kernel Goodness-of-Fit Test](https://mlsanity.com/paper/1705.07673/) (2017)
- [Learning deep kernels for exponential family densities](https://mlsanity.com/paper/1811.08357/) (2018)
- [Quadrature-based features for kernel approximation](https://mlsanity.com/paper/1802.03832/) (2018)
- [Random Fourier Features for Operator-Valued Kernels](https://mlsanity.com/paper/1605.02536/) (2016)
- [Estimation of High-Dimensional Graphical Models Using Regularized Score Matching](https://mlsanity.com/paper/1507.00433/) (2015)
- [Kernel Exponential Family Estimation via Doubly Dual Embedding](https://mlsanity.com/paper/1811.02228/) (2018)
- [Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families](https://mlsanity.com/paper/1506.02564/) (2015)
- [Efficient and principled score estimation with Nystr"om kernel exponential families](https://mlsanity.com/paper/1705.08360/) (2017)
- [Graphical Models for Non-Negative Data Using Generalized Score Matching](https://mlsanity.com/paper/1802.06340/) (2018)
- [A Unified Probabilistic Model for Learning Latent Factors and Their Connectivities from High-Dimensional Data](https://mlsanity.com/paper/1805.09567/) (2018)

## Built on by (most cited first, 1 of 1)

- [Early warning prediction: Onsager-Machlup vs Schr"odinger](https://mlsanity.com/paper/2602.00143/) (2026)

## Abstract (arXiv)

The density estimation is one of the core problems in statistics. Despite this, existing techniques like maximum likelihood estimation are computationally inefficient due to the intractability of the normalizing constant. For this reason an interest to score matching has increased being independent on the normalizing constant. However, such estimator is consistent only for distributions with the full space support. One of the approaches to make it consistent is to add noise to the input data which is called Denoising Score Matching. In this work we derive analytical expression for the Denoising Score matching using the Kernel Exponential Family as a model distribution. The usage of the kernel exponential family is motivated by the richness of this class of densities. To tackle the computational complexity we use Random Fourier Features based approximation of the kernel function. The analytical expression allows to drop additional regularization terms based on the higher-order derivatives as they are already implicitly included. Moreover, the obtained expression explicitly depends on the noise variance, so the validation loss can be straightforwardly used to tune the noise level. Along with benchmark experiments, the model was tested on various synthetic distributions to study the behaviour of the model in different cases. The empirical study shows comparable quality to the competing approaches, while the proposed method being computationally faster. The latter one enables scaling up to complex high-dimensional data.