# Learning Diffusion Priors from Observations by Expectation Maximization

arXiv: https://arxiv.org/abs/2405.13712 · HTML: https://mlsanity.com/paper/2405.13712/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 58 of its 109 references that are in the map.
- First posted on arXiv 22 May 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 58)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [Solving Inverse Problems in Medical Imaging with Score-Based Generative Models](https://mlsanity.com/paper/2111.08005/) (2021)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [Universal Guidance for Diffusion Models](https://mlsanity.com/paper/2302.07121/) (2023)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)

## Built on by (most cited first, 30 of 61)

- [A Survey on Diffusion Models for Inverse Problems](https://mlsanity.com/paper/2410.00083/) (2024)
- [On the Guidance of Flow Matching](https://mlsanity.com/paper/2502.02150/) (2025)
- [Generative Data Assimilation of Sparse Weather Station Observations at Kilometer Scales](https://mlsanity.com/paper/2406.16947/) (2024)
- [Distributional Diffusion Models with Scoring Rules](https://mlsanity.com/paper/2502.02483/) (2025)
- [Appa: Bending Weather Dynamics with Latent Diffusion Models for Global Data Assimilation](https://mlsanity.com/paper/2504.18720/) (2025)
- [Lost in Latent Space: An Empirical Study of Latent Diffusion Models for Physics Emulation](https://mlsanity.com/paper/2507.02608/) (2025)
- [How much is a noisy image worth? Data Scaling Laws for Ambient Diffusion](https://mlsanity.com/paper/2411.02780/) (2024)
- [Ambient Diffusion Omni: Training Good Models with Bad Data](https://mlsanity.com/paper/2506.10038/) (2025)
- [Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion](https://mlsanity.com/paper/2502.21278/) (2025)
- [Solving Inverse Problems with FLAIR](https://mlsanity.com/paper/2506.02680/) (2025)
- [RNE: plug-and-play diffusion inference-time control and energy-based training](https://mlsanity.com/paper/2506.05668/) (2025)
- [Diffusion models for inverse problems](https://mlsanity.com/paper/2508.01975/) (2025)
- [Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costs](https://mlsanity.com/paper/2410.11149/) (2024)
- [Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matching](https://mlsanity.com/paper/2406.10808/) (2024)
- [Frequency-Guided Posterior Sampling for Diffusion-Based Image Restoration](https://mlsanity.com/paper/2411.15295/) (2024)
- [LanPaint: Training-Free Diffusion Inpainting with Asymptotically Exact and Fast Conditional Sampling](https://mlsanity.com/paper/2502.03491/) (2025)
- [DiffEM: Learning from Corrupted Data with Diffusion Models via Expectation Maximization](https://mlsanity.com/paper/2510.12691/) (2025)
- [Generative Modeling from Black-box Corruptions via Self-Consistent Stochastic Interpolants](https://mlsanity.com/paper/2512.10857/) (2025)
- [Reward-Guided Discrete Diffusion via Clean-Sample Markov Chain for Molecule and Biological Sequence Design](https://mlsanity.com/paper/2602.09424/) (2026)
- [D-Flow SGLD: Source-Space Posterior Sampling for Scientific Inverse Problems with Flow Matching](https://mlsanity.com/paper/2602.21469/) (2026)
- [Inverse Flow and Consistency Models](https://mlsanity.com/paper/2502.11333/) (2025)
- [Training-Free Data Assimilation with GenCast](https://mlsanity.com/paper/2509.18811/) (2025)
- [DAISI: Data Assimilation with Inverse Sampling using Stochastic Interpolants](https://mlsanity.com/paper/2512.00252/) (2025)
- [Efficient Zero-Shot Inpainting with Decoupled Diffusion Guidance](https://mlsanity.com/paper/2512.18365/) (2025)
- [Self-Supervised Learning from Noisy and Incomplete Data](https://mlsanity.com/paper/2601.03244/) (2026)
- [Count Bridges enable Modeling and Deconvolving Transcriptomic Data](https://mlsanity.com/paper/2603.04730/) (2026)
- [Training-Free Bayesian Filtering with Generative Emulators](https://mlsanity.com/paper/2605.20028/) (2026)
- [Ambient Diffusion Policy: Imitation Learning from Suboptimal Data in Robotics](https://mlsanity.com/paper/2606.12365/) (2026)
- [PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inference](https://mlsanity.com/paper/2510.13763/) (2025)
- [WIND: Weather Inverse Diffusion for Zero-Shot Atmospheric Modeling](https://mlsanity.com/paper/2602.03924/) (2026)

## Abstract (arXiv)

Diffusion models recently proved to be remarkable priors for Bayesian inverse problems. However, training these models typically requires access to large amounts of clean data, which could prove difficult in some settings. In this work, we present DiEM, a novel method based on the expectation-maximization algorithm for training diffusion models from incomplete and noisy observations only. Unlike previous works, DiEM leads to proper diffusion models, which is crucial for downstream tasks. As part of our methods, we propose and motivate an improved posterior sampling scheme for unconditional diffusion models. We present empirical evidence supporting the effectiveness of our approach.