# Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data

arXiv: https://arxiv.org/abs/2404.10177 · HTML: https://mlsanity.com/paper/2404.10177/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 33 of its 43 references that are in the map.
- First posted on arXiv 20 Mar 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 33)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [DataComp: In search of the next generation of multimodal datasets](https://mlsanity.com/paper/2304.14108/) (2023)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [Noise2Void - Learning Denoising from Single Noisy Images](https://mlsanity.com/paper/1811.10980/) (2018)
- [Noise2Self: Blind Denoising by Self-Supervision](https://mlsanity.com/paper/1901.11365/) (2019)
- [Understanding and Mitigating Copying in Diffusion Models](https://mlsanity.com/paper/2305.20086/) (2023)
- [Noisier2Noise: Learning to Denoise from Unpaired Noisy Data](https://mlsanity.com/paper/1910.11908/) (2019)
- [Ambient Diffusion: Learning Clean Distributions from Corrupted Data](https://mlsanity.com/paper/2305.19256/) (2023)
- [Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistent](https://mlsanity.com/paper/2302.09057/) (2023)
- [DDM2: Self-Supervised Diffusion MRI Denoising with Generative Diffusion Models](https://mlsanity.com/paper/2302.03018/) (2023)
- [Target Score Matching](https://mlsanity.com/paper/2402.08667/) (2024)
- [Unsupervised Learning with Stein's Unbiased Risk Estimator](https://mlsanity.com/paper/1805.10531/) (2018)
- [Training Deep Learning Based Denoisers without Ground Truth Data](https://mlsanity.com/paper/1803.01314/) (2018)
- [Solving Inverse Problems with Score-Based Generative Priors learned from Noisy Data](https://mlsanity.com/paper/2305.01166/) (2023)
- [GSURE-Based Diffusion Model Training with Corrupted Data](https://mlsanity.com/paper/2305.13128/) (2023)
- [FP-Diffusion: Improving Score-based Diffusion Models by Enforcing the Underlying Score Fokker-Planck Equation](https://mlsanity.com/paper/2210.04296/) (2022)

## Built on by (most cited first, 30 of 44)

- [A Survey on Diffusion Models for Inverse Problems](https://mlsanity.com/paper/2410.00083/) (2024)
- [Learning Diffusion Priors from Observations by Expectation Maximization](https://mlsanity.com/paper/2405.13712/) (2024)
- [Be like a Goldfish, Don't Memorize! Mitigating Memorization in Generative LLMs](https://mlsanity.com/paper/2406.10209/) (2024)
- [A Geometric Framework for Understanding Memorization in Generative Models](https://mlsanity.com/paper/2411.00113/) (2024)
- [Warped Diffusion: Solving Video Inverse Problems with Image Diffusion Models](https://mlsanity.com/paper/2410.16152/) (2024)
- [Classifier-Free Guidance inside the Attraction Basin May Cause Memorization](https://mlsanity.com/paper/2411.16738/) (2024)
- [An Expectation-Maximization Algorithm for Training Clean Diffusion Models from Corrupted Observations](https://mlsanity.com/paper/2407.01014/) (2024)
- [How much is a noisy image worth? Data Scaling Laws for Ambient Diffusion](https://mlsanity.com/paper/2411.02780/) (2024)
- [Ambient Diffusion Omni: Training Good Models with Bad Data](https://mlsanity.com/paper/2506.10038/) (2025)
- [Flow Matching: Markov Kernels, Stochastic Processes and Transport Plans](https://mlsanity.com/paper/2501.16839/) (2025)
- [Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion](https://mlsanity.com/paper/2502.21278/) (2025)
- [Diffusion models for inverse problems](https://mlsanity.com/paper/2508.01975/) (2025)
- [Replication in Visual Diffusion Models: A Survey and Outlook](https://mlsanity.com/paper/2408.00001/) (2024)
- [Stochastic Forward-Backward Deconvolution: Training Diffusion Models with Finite Noisy Datasets](https://mlsanity.com/paper/2502.05446/) (2025)
- [Disguised Copyright Infringement of Latent Diffusion Models](https://mlsanity.com/paper/2404.06737/) (2024)
- [Score-based Self-supervised MRI Denoising](https://mlsanity.com/paper/2505.05631/) (2025)
- [Measurement Score-Based Diffusion Model](https://mlsanity.com/paper/2505.11853/) (2025)
- [Reinforcing Few-step Generators via Reward-Tilted Distribution Matching](https://mlsanity.com/paper/2605.26108/) (2026)
- [Generate and Instantiate What You Prefer: Text-Guided Diffusion for Sequential Recommendation](https://mlsanity.com/paper/2410.13428/) (2024)
- [Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High-Quality Generation](https://mlsanity.com/paper/2503.07578/) (2025)
- [Random Walks with Tweedie: A Unified View of Score-Based Diffusion Models](https://mlsanity.com/paper/2411.18702/) (2024)
- [DiffEM: Learning from Corrupted Data with Diffusion Models via Expectation Maximization](https://mlsanity.com/paper/2510.12691/) (2025)
- [X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrations](https://mlsanity.com/paper/2511.04671/) (2025)
- [Generative Modeling from Black-box Corruptions via Self-Consistent Stochastic Interpolants](https://mlsanity.com/paper/2512.10857/) (2025)
- [Inverse Flow and Consistency Models](https://mlsanity.com/paper/2502.11333/) (2025)
- [Robust multi-coil MRI reconstruction via self-supervised denoising](https://mlsanity.com/paper/2411.12919/) (2024)
- [SolidMark: Evaluating Image Memorization in Generative Models](https://mlsanity.com/paper/2503.00592/) (2025)
- [Normalization-equivariant Diffusion Models: Learning Posterior Samplers From Noisy And Partial Measurements](https://mlsanity.com/paper/2510.11964/) (2025)
- [ADG: Ambient Diffusion-Guided Dataset Recovery for Corruption-Robust Offline Reinforcement Learning](https://mlsanity.com/paper/2505.23871/) (2025)
- [SFBD-OMNI: Bridge models for lossy measurement restoration with limited clean samples](https://mlsanity.com/paper/2512.17051/) (2025)

## Abstract (arXiv)

Ambient diffusion is a recently proposed framework for training diffusion models using corrupted data. Both Ambient Diffusion and alternative SURE-based approaches for learning diffusion models from corrupted data resort to approximations which deteriorate performance. We present the first framework for training diffusion models that provably sample from the uncorrupted distribution given only noisy training data, solving an open problem in this space. Our key technical contribution is a method that uses a double application of Tweedie's formula and a consistency loss function that allows us to extend sampling at noise levels below the observed data noise. We also provide further evidence that diffusion models memorize from their training sets by identifying extremely corrupted images that are almost perfectly reconstructed, raising copyright and privacy concerns. Our method for training using corrupted samples can be used to mitigate this problem. We demonstrate this by fine-tuning Stable Diffusion XL to generate samples from a distribution using only noisy samples. Our framework reduces the amount of memorization of the fine-tuning dataset, while maintaining competitive performance.