mlsanityData to 7 Oct 2026

Denoising Deep Generative Models

arXiv 2212.01265 ↗First posted 30 Nov 2022cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Likelihood-based deep generative models have recently been shown to exhibit pathological behaviour under the manifold hypothesis as a consequence of using high-dimensional densities to model data with low-dimensional structure. In this paper we propose two methodologies aimed at addressing this problem. Both are based on adding Gaussian noise to the data to remove the dimensionality mismatch during training, and both provide a denoising mechanism whose goal is to sample from the model as though no noise had been added to the data. Our first approach is based on Tweedie's formula, and the second on models which take the variance of added noise as a conditional input. We show that surprisingly, while well motivated, these approaches only sporadically improve performance over not adding noise, and that other methods of addressing the dimensionality mismatch are more empirically adequate.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 44 of its 63 references that are in the map. First posted on arXiv 30 Nov 2022.

Builds on

44 of 63 references.

References · 44

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2013Auto-Encoding Variational Bayescited by 3,980
  7. 2016Density estimation using Real NVPcited by 2,029
  8. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  9. 2016Pixel Recurrent Neural Networkscited by 814
  10. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  11. 2019Neural Spline Flowscited by 590
  12. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
32 more
  1. 2018Invertible Residual Networkscited by 364
  2. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 307
  3. 2015A note on the evaluation of generative modelscited by 291
  4. 2019Residual Flows for Invertible Generative Modelingcited by 256
  5. 2022Convergence of denoising diffusion models under the manifold hypothesiscited by 237
  6. 2022Riemannian Score-Based Generative Modellingcited by 216
  7. 2019Diagnosing and Enhancing VAE Modelscited by 192
  8. 2019Learning Likelihoods with Conditional Normalizing Flowscited by 170
  9. 2019Guided Image Generation with Conditional Invertible Neural Networkscited by 147
  10. 2020Riemannian Continuous Normalizing Flowscited by 143
  11. 2022Score-Based Generative Models Detect Manifoldscited by 121
  12. 2020Normalizing Flows on Tori and Spherescited by 117
  13. 2020Flows for simultaneous manifold learning and density estimationcited by 117
  14. 2020SoftFlow: Probabilistic Framework for Normalizing Flow on Manifoldscited by 97
  15. 2019Robust and interpretable blind image denoising via bias-free convolutional neural networkscited by 87
  16. 2020Generalized Energy Based Modelscited by 73
  17. 2016Normalizing Flows on Riemannian Manifoldscited by 68
  18. 2013RNADE: The real-valued neural autoregressive density-estimatorcited by 65
  19. 2021Estimating High Order Gradients of the Data Distribution by Denoisingcited by 63
  20. 2021Rectangular Flows for Manifold Learningcited by 49
  21. 2019Neural Empirical Bayescited by 46
  22. 2021Trumpets: Injective Flows for Inference and Inverse Problemscited by 36
  23. 2015Generative Image Modeling Using Spatial LSTMscited by 35
  24. 2021Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flowscited by 30
  25. 2022LIDL: Local Intrinsic Dimension Estimation Using Approximate Likelihoodcited by 30
  26. 2022Diagnosing and Fixing Manifold Overfitting in Deep Generative Modelscited by 28
  27. 2018Spread Divergencecited by 23
  28. 2021Improved Autoregressive Modeling with Distribution Smoothingcited by 18
  29. 2019Semi-Conditional Normalizing Flows for Semi-Supervised Learningcited by 15
  30. 2021A likelihood approach to nonparametric estimation of a singular distribution using deep generative modelscited by 12
  31. 2021Density estimation on low-dimensional manifolds: an inflation-deflation approachcited by 11
  32. 2016Deep Variational Inference Without Pixel-Wise Reconstructioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 1; 2025: 1; 2026: 1.

6 papers.

  1. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  2. 2023Lifting Architectural Constraints of Injective Flowscited by 17
  3. 2023A Review of Change of Variable Formulas for Generative Modelingcited by 12
  4. 2023One-Line-of-Code Data Mollification Improves Optimization of Likelihood-based Generative Modelscited by 4
  5. 2025On Convolutions, Intrinsic Dimension, and Diffusion Modelscited by 2
  6. 2026From Core to Detail: Unsupervised Disentanglement with Entropy-Ordered Flowscited by 0