mlsanityData to 7 Oct 2026

Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data

arXiv 2404.10177 ↗First posted 20 Mar 2024cs.CV cs.AI cs.LGcited by 44

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Ambient diffusion is a recently proposed framework for training diffusion models using corrupted data. Both Ambient Diffusion and alternative SURE-based approaches for learning diffusion models from corrupted data resort to approximations which deteriorate performance. We present the first framework for training diffusion models that provably sample from the uncorrupted distribution given only noisy training data, solving an open problem in this space. Our key technical contribution is a method that uses a double application of Tweedie's formula and a consistency loss function that allows us to extend sampling at noise levels below the observed data noise. We also provide further evidence that diffusion models memorize from their training sets by identifying extremely corrupted images that are almost perfectly reconstructed, raising copyright and privacy concerns. Our method for training using corrupted samples can be used to mitigate this problem. We demonstrate this by fine-tuning Stable Diffusion XL to generate samples from a distribution using only noisy samples. Our framework reduces the amount of memorization of the fine-tuning dataset, while maintaining competitive performance.

Key facts. Cited by 44 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 33 of its 43 references that are in the map. First posted on arXiv 20 Mar 2024.

Builds on

33 of 43 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Target Score Matching13 Feb
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  3. 2023On the Equivalence of Consistency-Type Models: Consistency Models, Consistent Diffusion Models, and Fokker-Planck Regularization1 Jun
  4. 2023Understanding and Mitigating Copying in Diffusion Models31 May
  5. 2023GSURE-Based Diffusion Model Training with Corrupted Data22 May
  6. 2023Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistent17 Feb
  7. 2022FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation9 Oct
  8. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  10. 2020Denoising Diffusion Implicit Models6 Oct
  11. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 22

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2022Generative Adversarial Networkscited by 5,603
  3. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2020Extracting Training Data from Large Language Modelscited by 2,144
  7. 2022Denoising Diffusion Restoration Modelscited by 951
  8. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  9. 2023Extracting Training Data from Diffusion Modelscited by 723
  10. 2023DataComp: In search of the next generation of multimodal datasetscited by 635
  11. 2018Noise2Noise: Learning Image Restoration without Clean Datacited by 526
  12. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
10 more
  1. 2018Noise2Void - Learning Denoising from Single Noisy Imagescited by 359
  2. 2019Noise2Self: Blind Denoising by Self-Supervisioncited by 274
  3. 2019Noisier2Noise: Learning to Denoise from Unpaired Noisy Datacited by 125
  4. 2023Ambient Diffusion: Learning Clean Distributions from Corrupted Datacited by 108
  5. 2023DDM2: Self-Supervised Diffusion MRI Denoising with Generative Diffusion Modelscited by 50
  6. 2018Unsupervised Learning with Stein's Unbiased Risk Estimatorcited by 47
  7. 2018Training Deep Learning Based Denoisers without Ground Truth Datacited by 43
  8. 2023Solving Inverse Problems with Score-Based Generative Priors learned from Noisy Datacited by 40
  9. 2022Self-Consistency of the Fokker-Planck Equationcited by 18
  10. 2020ENSURE: A General Approach for Unsupervised Training of Deep Image Reconstruction Algorithmscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 20; 2026: 10.

44 papers.

Sort:

By area:

  1. 2024A Survey on Diffusion Models for Inverse Problemscited by 173
  2. 2024Learning Diffusion Priors from Observations by Expectation Maximizationcited by 60
  3. 2024Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMscited by 50
  4. 2024A Geometric Framework for Understanding Memorization in Generative Modelscited by 40
  5. 2024Warped Diffusion: Solving Video Inverse Problems with Image Diffusion Modelscited by 21
  6. 2024Classifier-Free Guidance inside the Attraction Basin May Cause Memorizationcited by 21
  7. 2024An Expectation-Maximization Algorithm for Training Clean Diffusion Models from Corrupted Observationscited by 20
  8. 2024How much is a noisy image worth? Data Scaling Laws for Ambient Diffusioncited by 18
  9. 2025Ambient Diffusion Omni: Training Good Models with Bad Datacited by 18
  10. 2025Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusioncited by 17
  11. 2025Flow Matching: Markov Kernels, Stochastic Processes and Transport Planscited by 16
  12. 2025Diffusion models for inverse problemscited by 15
32 more
  1. 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
  2. 2025Stochastic Forward-Backward Deconvolution: Training Diffusion Models with Finite Noisy Datasetscited by 11
  3. 2024Disguised Copyright Infringement of Latent Diffusion Modelscited by 10
  4. 2025Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High-Quality Generationcited by 7
  5. 2025Measurement Score-Based Diffusion Modelcited by 7
  6. 2024Generate and Instantiate What You Prefer: Text-Guided Diffusion for Sequential Recommendationcited by 6
  7. 2024Random Walks with Tweedie: A Unified View of Score-Based Diffusion Modelscited by 6
  8. 2025Score-based Self-supervised MRI Denoisingcited by 6
  9. 2025DiffEM: Learning from Corrupted Data with Diffusion Models via Expectation Maximizationcited by 6
  10. 2025X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrationscited by 6
  11. 2026Reinforcing Few-step Generators via Reward-Tilted Distribution Matchingcited by 6
  12. 2025Generative Modeling from Black-box Corruptions via Self-Consistent Stochastic Interpolantscited by 5
  13. 2024Robust multi-coil MRI reconstruction via self-supervised denoisingcited by 4
  14. 2025Inverse Flow and Consistency Modelscited by 4
  15. 2025SolidMark: Evaluating Image Memorization in Generative Modelscited by 3
  16. 2025Normalization-equivariant Diffusion Models: Learning Posterior Samplers From Noisy And Partial Measurementscited by 3
  17. 2025ADG: Ambient Diffusion-Guided Dataset Recovery for Corruption-Robust Offline Reinforcement Learningcited by 2
  18. 2025SFBD-OMNI: Bridge models for lossy measurement restoration with limited clean samplescited by 2
  19. 2026Ambient Dataloops: Generative Models for Dataset Refinementcited by 2
  20. 2026SCoRe: Clean Image Generation from Diffusion Models Trained on Noisy Imagescited by 2
  21. 2025SFBD Flow: A Continuous-Optimization Framework for Training Diffusion Models with Noisy Samplescited by 1
  22. 2026Riemannian AmbientFlow: Towards Simultaneous Manifold Learning and Generative Modeling from Corrupted Datacited by 1
  23. 2026Optimizing Diffusion Priors in Image Reconstruction from a Single Observationcited by 1
  24. 2024Opt-In Art: Learning Art Styles Only from Few Examplescited by 0
  25. 2025Bootstrapping Diffusion: Diffusion Model Training Leveraging Partial and Corrupted Datacited by 0
  26. 2025What is Adversarial Training for Diffusion Models?cited by 0
  27. 2025UPMRI: Unsupervised Parallel MRI Reconstruction via Projected Conditional Flow Matchingcited by 0
  28. 2026Sampling via Stochastic Interpolants by Langevin-based Velocity and Initialization Estimation in Flow ODEscited by 0
  29. 2026Memorization In Stable Diffusion Is Unexpectedly Driven by CLIP Embeddingscited by 0
  30. 2026Reflection Separation from a Single Image via Joint Latent Diffusioncited by 0
  31. 2026Noise-Robust Conditional Flow Matching: Generating Clean Samples from Noisy Datasetscited by 0
  32. 2026IP Protection in the Era of Visual Generative AI: A Surveycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023The Emergence of Reproducibility and Generalizability in Diffusion Models
  2. 2025Quantifying the Ease of Reproducing Training Data in Unconditional Diffusion Models
  3. 2025Diffusion Models: A Mathematical Introduction
  4. 2024Towards a mathematical theory for consistency training in diffusion models
  5. 2023The Journey, Not the Destination: How Data Guides Diffusion Models
  6. 2024Towards Faster Training of Diffusion Models: An Inspiration of A Consistency Phenomenon
  7. 2023Stable Target Field for Reduced Variance Score Estimation in Diffusion Models
  8. 2023Unmasking Bias in Diffusion Model Training