mlsanityData to 7 Oct 2026

Ambient Diffusion: Learning Clean Distributions from Corrupted Data

arXiv 2305.19256 ↗First posted 30 May 2023cs.LG cs.AI cs.CVcited by 108

Not on the map. Why?

Cited by 66 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present the first diffusion-based framework that can learn an unknown distribution using only highly-corrupted samples. This problem arises in scientific applications where access to uncorrupted samples is impossible or expensive to acquire. Another benefit of our approach is the ability to train generative models that are less likely to memorize individual training samples since they never observe clean training data. Our main idea is to introduce additional measurement distortion during the diffusion process and require the model to predict the original corrupted image from the further corrupted image. We prove that our method leads to models that learn the conditional expectation of the full uncorrupted image given this additional measurement corruption. This holds for any corruption process that satisfies some technical conditions (and in particular includes inpainting and compressed sensing). We train models on standard benchmarks (CelebA, CIFAR-10 and AFHQ) and show that we can learn the distribution even when all the training samples have 90% of their pixels missing. We also show that we can finetune foundation models on small corrupted datasets (e.g. MRI scans with block corruptions) and learn the clean distribution without memorizing the training set.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Builds on 50 of its 64 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

50 of 64 references.

References · 50

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  10. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  12. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
38 more
  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  2. 2020Extracting Training Data from Large Language Modelscited by 2,144
  3. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  4. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  5. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  6. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  8. 2022Denoising Diffusion Restoration Modelscited by 951
  9. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  10. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  11. 2018Noise2Noise: Learning Image Restoration without Clean Datacited by 526
  12. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptionscited by 440
  13. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
  14. 2018Noise2Void - Learning Denoising from Single Noisy Imagescited by 359
  15. 2020PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Modelscited by 358
  16. 2021Robust Compressed Sensing MRI with Deep Generative Priorscited by 284
  17. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noisecited by 275
  18. 2018Toward Convolutional Blind Denoising of Real Photographscited by 274
  19. 2019Noise2Self: Blind Denoising by Self-Supervisioncited by 274
  20. 2021SNIPS: Solving Noisy Inverse Problems Stochasticallycited by 191
  21. 2023Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restorationcited by 164
  22. 2022Measuring Forgetting of Memorized Training Examplescited by 118
  23. 2022Soft Diffusion: Score Matching for General Corruptionscited by 101
  24. 2019Coherent Semantic Attention for Image Inpaintingcited by 97
  25. 2021Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimationcited by 95
  26. 2020Solving Linear Inverse Problems Using the Prior Implicit in a Denoisercited by 78
  27. 2019MisGAN: Learning from Incomplete Data with Generative Adversarial Networkscited by 74
  28. 2018A Poisson-Gaussian Denoising Dataset with Real Fluorescence Microscopy Imagescited by 70
  29. 2020not-MIWAE: Deep Generative Modelling with Missing not at Random Datacited by 61
  30. 2023Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistentcited by 58
  31. 2023Solving Inverse Problems with Score-Based Generative Priors learned from Noisy Datacited by 40
  32. 2023GSURE-Based Diffusion Model Training with Corrupted Datacited by 40
  33. 2020MCFlow: Monte Carlo Flow Models for Data Imputationcited by 37
  34. 2022A theoretical framework for self-supervised MR image reconstruction using sub-sampling via variable density Noisier2Noisecited by 29
  35. 2022Self-Score: Self-Supervised Learning on Score-Based Models for MRI Reconstructioncited by 23
  36. 2020Review: Deep Learning in Electron Microscopycited by 20
  37. 2022Unsupervised Learning From Incomplete Measurements for Inverse Problemscited by 20
  38. 2023Image Reconstruction without Explicit Priorscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 8; 2024: 33; 2025: 42; 2026: 25.

108 papers.

Sort:

By area:

  1. 2024A Survey on Diffusion Models for Inverse Problemscited by 173
  2. 2024Detecting, Explaining, and Mitigating Memorization in Diffusion Modelscited by 94
  3. 2024Learning Diffusion Priors from Observations by Expectation Maximizationcited by 60
  4. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  5. 2024Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attentioncited by 57
  6. 2024Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMscited by 50
  7. 2024Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Datacited by 44
  8. 2023MissDiff: Training Diffusion Models on Tabular Data with Missing Valuescited by 38
  9. 2023Taming Mode Collapse in Score Distillation for Text-to-3D Generationcited by 35
  10. 2024Ambient Diffusion Posterior Sampling: Solving Inverse Problems with Diffusion Models Trained on Corrupted Datacited by 32
  11. 2025On the Edge of Memorization in Diffusion Modelscited by 22
  12. 2024Deep Diffusion Image Prior for Efficient OOD Adaptation in 3D Inverse Problemscited by 21
38 more
  1. 2024Warped Diffusion: Solving Video Inverse Problems with Image Diffusion Modelscited by 21
  2. 2024An Expectation-Maximization Algorithm for Training Clean Diffusion Models from Corrupted Observationscited by 20
  3. 2024Exploring Local Memorization in Diffusion Models via Bright Ending Attentioncited by 19
  4. 2024How much is a noisy image worth? Data Scaling Laws for Ambient Diffusioncited by 18
  5. 2025Ambient Diffusion Omni: Training Good Models with Bad Datacited by 18
  6. 2025Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusioncited by 17
  7. 2025Taking a Big Step: Large Learning Rates in Denoising Score Matching Prevent Memorizationcited by 16
  8. 2023Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstructioncited by 15
  9. 2025Diffusion models for inverse problemscited by 15
  10. 2024Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimationcited by 13
  11. 2024Osmosis: RGBD Diffusion Prior for Underwater Image Restorationcited by 13
  12. 2024Generalized Recorrupted-to-Recorrupted: Self-Supervised Learning Beyond Gaussian Noisecited by 13
  13. 2024Integrating Amortized Inference with Diffusion Models for Learning Clean Distribution from Corrupted Imagescited by 12
  14. 2024Iterative Ensemble Training with Anti-Gradient Control for Mitigating Memorization in Diffusion Modelscited by 12
  15. 2023AmbientFlow: Invertible generative models from incomplete, noisy measurementscited by 11
  16. 2024Efficient Prior Calibration From Indirect Datacited by 11
  17. 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
  18. 2025Stochastic Forward-Backward Deconvolution: Training Diffusion Models with Finite Noisy Datasetscited by 11
  19. 2024MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generationcited by 10
  20. 2025Motion Synthesis with Sparse and Flexible Keyjoint Controlcited by 9
  21. 2025Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High-Quality Generationcited by 7
  22. 2025Measurement Score-Based Diffusion Modelcited by 7
  23. 2025Score Matching With Missing Datacited by 7
  24. 2024Amortized Posterior Sampling with Diffusion Prior Distillationcited by 6
  25. 2025A Diffusion-Based Framework for Terrain-Aware Remote Sensing Image Reconstructioncited by 6
  26. 2025How Diffusion Models Memorizecited by 6
  27. 2025DiffEM: Learning from Corrupted Data with Diffusion Models via Expectation Maximizationcited by 6
  28. 2025X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrationscited by 6
  29. 2023K-band: Self-supervised MRI Reconstruction via Stochastic Gradient Descent over K-space Subsetscited by 5
  30. 2024DP-TLDM: Differentially Private Tabular Latent Diffusion Modelcited by 5
  31. 2024Perspective-Equivariance for Unsupervised Imaging with Camera Geometrycited by 5
  32. 2024MemBench: Memorized Image Trigger Prompt Dataset for Diffusion Modelscited by 5
  33. 2024Investigating Memorization in Video Diffusion Modelscited by 5
  34. 2025StableMotion: Training Motion Cleanup Models with Unpaired Corrupted Datacited by 5
  35. 2025Diagnosing and Improving Diffusion Models by Estimating the Optimal Loss Valuecited by 5
  36. 2025Generative Modeling from Black-box Corruptions via Self-Consistent Stochastic Interpolantscited by 5
  37. 2026Self-Supervised Learning from Noisy and Incomplete Datacited by 5
  38. 2024Robust multi-coil MRI reconstruction via self-supervised denoisingcited by 4

50 most cited of 108 All 108 (JSON)