# Stochastic Forward-Backward Deconvolution: Training Diffusion Models with Finite Noisy Datasets

arXiv: https://arxiv.org/abs/2502.05446 · HTML: https://mlsanity.com/paper/2502.05446/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 35 of its 63 references that are in the map.
- First posted on arXiv 8 Feb 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 35)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [Diffusion Models for Adversarial Purification](https://mlsanity.com/paper/2205.07460/) (2022)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [Diffsound: Discrete Diffusion Model for Text-to-sound Generation](https://mlsanity.com/paper/2207.09983/) (2022)
- [Differentially Private Generative Adversarial Network](https://mlsanity.com/paper/1802.06739/) (2018)
- [Diffusion-GAN: Training GANs with Diffusion](https://mlsanity.com/paper/2206.02262/) (2022)
- [Solving Schr"odinger Bridges via Maximum Likelihood](https://mlsanity.com/paper/2106.02081/) (2021)
- [Differentially Private Diffusion Models](https://mlsanity.com/paper/2210.09929/) (2022)
- [Ambient Diffusion: Learning Clean Distributions from Corrupted Data](https://mlsanity.com/paper/2305.19256/) (2023)
- [Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistent](https://mlsanity.com/paper/2302.09057/) (2023)
- [Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data](https://mlsanity.com/paper/2404.10177/) (2024)
- [Solving Inverse Problems with Score-Based Generative Priors learned from Noisy Data](https://mlsanity.com/paper/2305.01166/) (2023)

## Built on by (most cited first, 11 of 11)

- [Denoising Score Distillation: From Noisy Diffusion Pretraining to One-Step High-Quality Generation](https://mlsanity.com/paper/2503.07578/) (2025)
- [Generative Modeling from Black-box Corruptions via Self-Consistent Stochastic Interpolants](https://mlsanity.com/paper/2512.10857/) (2025)
- [Normalization-equivariant Diffusion Models: Learning Posterior Samplers From Noisy And Partial Measurements](https://mlsanity.com/paper/2510.11964/) (2025)
- [Blind Strong Gravitational Lensing Inversion: Joint Inference of Source and Lens Mass with Score-Based Models](https://mlsanity.com/paper/2511.04792/) (2025)
- [SFBD-OMNI: Bridge models for lossy measurement restoration with limited clean samples](https://mlsanity.com/paper/2512.17051/) (2025)
- [Ambient Dataloops: Generative Models for Dataset Refinement](https://mlsanity.com/paper/2601.15417/) (2026)
- [SFBD Flow: A Continuous-Optimization Framework for Training Diffusion Models with Noisy Samples](https://mlsanity.com/paper/2506.02371/) (2025)
- [Riemannian AmbientFlow: Towards Simultaneous Manifold Learning and Generative Modeling from Corrupted Data](https://mlsanity.com/paper/2601.18728/) (2026)
- [MAD: Manifold Attracted Diffusion](https://mlsanity.com/paper/2509.24710/) (2025)
- [Nonparametric Deconvolution and Denoising using Simulation Based Inference](https://mlsanity.com/paper/2606.21907/) (2026)
- [Noise-Robust Conditional Flow Matching: Generating Clean Samples from Noisy Datasets](https://mlsanity.com/paper/2608.00064/) (2026)

## Abstract (arXiv)

Recent diffusion-based generative models achieve remarkable results by training on massive datasets, yet this practice raises concerns about memorization and copyright infringement. A proposed remedy is to train exclusively on noisy data with potential copyright issues, ensuring the model never observes original content. However, through the lens of deconvolution theory, we show that although it is theoretically feasible to learn the data distribution from noisy samples, the practical challenge of collecting sufficient samples makes successful learning nearly unattainable. To overcome this limitation, we propose to pretrain the model with a small fraction of clean data to guide the deconvolution process. Combined with our Stochastic Forward--Backward Deconvolution (SFBD) method, we attain FID 6.31 on CIFAR-10 with just 4% clean images (and 3.58 with 10%). We also provide theoretical guarantees that SFBD learns the true data distribution. These results underscore the value of limited clean pretraining, or pretraining on similar datasets. Empirical studies further validate and enrich our findings.