# Learning Energy-Based Models in High-Dimensional Spaces with Multi-scale Denoising Score Matching

arXiv: https://arxiv.org/abs/1910.07762 · HTML: https://mlsanity.com/paper/1910.07762/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2019.

## Built on by (most cited first, 26 of 26)

- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)
- [MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolation](https://mlsanity.com/paper/2205.09853/) (2022)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglement](https://mlsanity.com/paper/2301.03028/) (2023)
- [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023)
- [Energy-Based Transformers are Scalable Learners and Thinkers](https://mlsanity.com/paper/2507.02092/) (2025)
- [Entropy-based Training Methods for Scalable Neural Implicit Sampler](https://mlsanity.com/paper/2306.04952/) (2023)
- [Aligning Diffusion Behaviors with Q-functions for Efficient Continuous Control](https://mlsanity.com/paper/2407.09024/) (2024)
- [X-GGM: Graph Generative Modeling for Out-of-Distribution Generalization in Visual Question Answering](https://mlsanity.com/paper/2107.11576/) (2021)
- [Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction](https://mlsanity.com/paper/2309.00073/) (2023)
- [Energy Discrepancies: A Score-Independent Loss for Energy-Based Models](https://mlsanity.com/paper/2307.06431/) (2023)
- [Multimeasurement Generative Models](https://mlsanity.com/paper/2112.09822/) (2021)
- [Score Matching With Missing Data](https://mlsanity.com/paper/2506.00557/) (2025)
- [Follow the Energy, Find the Path: Riemannian Metrics from Energy-Based Models](https://mlsanity.com/paper/2505.18230/) (2025)
- [Bi-level Doubly Variational Learning for Energy-based Latent Variable Models](https://mlsanity.com/paper/2203.14702/) (2022)
- [Score-based Generative Priors Guided Model-driven Network for MRI Reconstruction](https://mlsanity.com/paper/2405.02958/) (2024)
- [EBMs Trained with Maximum Likelihood are Generator Models Trained with a Self-adverserial Loss](https://mlsanity.com/paper/2102.11757/) (2021)
- [How to Construct Energy for Images? Denoising Autoencoder Can Be Energy Based Model](https://mlsanity.com/paper/2303.03887/) (2023)
- [Offline Reinforcement Learning with Penalized Action Noise Injection](https://mlsanity.com/paper/2507.02356/) (2025)
- [Denoising Fisher Training For Neural Implicit Samplers](https://mlsanity.com/paper/2411.01453/) (2024)
- [Deep End-to-End Posterior ENergy (DEEPEN) for image recovery](https://mlsanity.com/paper/2503.17244/) (2025)
- [Motion-compensated cardiac MRI using low-rank diffeomorphic flow (DMoCo)](https://mlsanity.com/paper/2505.03149/) (2025)
- [Scaling Non-Parametric Sampling with Representation](https://mlsanity.com/paper/2510.22196/) (2025)
- [Annealed Langevin Posterior Sampling (ALPS): A Rapid Algorithm for Image Restoration with Multiscale Energy Models](https://mlsanity.com/paper/2601.02594/) (2026)
- [Learning Unified Representations of Normalcy for Time Series Anomaly Detection](https://mlsanity.com/paper/2605.09685/) (2026)
- [Composing Flow-Matching Energies with Known Physics: Generation, OOD Detection, and Inversion on PDE Fields](https://mlsanity.com/paper/2608.18004/) (2026)

## Abstract (arXiv)

Energy-Based Models (EBMs) assign unnormalized log-probability to data samples. This functionality has a variety of applications, such as sample synthesis, data denoising, sample restoration, outlier detection, Bayesian reasoning, and many more. But training of EBMs using standard maximum likelihood is extremely slow because it requires sampling from the model distribution. Score matching potentially alleviates this problem. In particular, denoising score matching citepvincent2011connection has been successfully used to train EBMs. Using noisy data samples with one fixed noise level, these models learn fast and yield good results in data denoising citepsaremi2019neural. However, demonstrations of such models in high quality sample synthesis of high dimensional data were lacking. Recently, citetsong2019generative have shown that a generative model trained by denoising score matching accomplishes excellent sample synthesis, when trained with data samples corrupted with multiple levels of noise. Here we provide analysis and empirical evidence showing that training with multiple noise levels is necessary when the data dimension is high. Leveraging this insight, we propose a novel EBM trained with multi-scale denoising score matching. Our model exhibits data generation performance comparable to state-of-the-art techniques such as GANs, and sets a new baseline for EBMs. The proposed model also provides density information and performs well in an image inpainting task.