# Ensemble Kalman Diffusion Guidance: A Derivative-free Method for Inverse Problems

arXiv: https://arxiv.org/abs/2409.20175 · HTML: https://mlsanity.com/paper/2409.20175/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 37 of its 64 references that are in the map.
- First posted on arXiv 30 Sep 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 37)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [Solving Inverse Problems in Medical Imaging with Score-Based Generative Models](https://mlsanity.com/paper/2111.08005/) (2021)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023)
- [Denoising Diffusion Models for Plug-and-Play Image Restoration](https://mlsanity.com/paper/2305.08995/) (2023)
- [A Variational Perspective on Solving Inverse Problems with Diffusion Models](https://mlsanity.com/paper/2305.04391/) (2023)
- [Fast Sampling of Diffusion Models via Operator Learning](https://mlsanity.com/paper/2211.13449/) (2022)
- [Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models](https://mlsanity.com/paper/2307.00619/) (2023)
- [Parallel Diffusion Models of Operator and Image for Blind Inverse Problems](https://mlsanity.com/paper/2211.10656/) (2022)
- [Principled Probabilistic Imaging using Diffusion Models as Plug-and-Play Priors](https://mlsanity.com/paper/2405.18782/) (2024)
- [Classifier-Free Guidance is a Predictor-Corrector](https://mlsanity.com/paper/2408.09000/) (2024)
- [Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusion](https://mlsanity.com/paper/2312.00852/) (2023)
- [Prompt-tuning latent diffusion models for inverse problems](https://mlsanity.com/paper/2310.01110/) (2023)
- [A Theoretical and Empirical Comparison of Gradient Approximations in Derivative-Free Optimization](https://mlsanity.com/paper/1905.01332/) (2019)
- [Deep Probabilistic Imaging: Uncertainty Quantification and Multi-modal Solution Characterization for Computational Imaging](https://mlsanity.com/paper/2010.14462/) (2020)
- [Symbolic Music Generation with Non-Differentiable Rule Guided Diffusion](https://mlsanity.com/paper/2402.14285/) (2024)
- [Provable Probabilistic Imaging using Score-Based Generative Priors](https://mlsanity.com/paper/2310.10835/) (2023)
- [RB-Modulation: Training-Free Personalization of Diffusion Models using Stochastic Optimal Control](https://mlsanity.com/paper/2405.17401/) (2024)
- [Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance](https://mlsanity.com/paper/2402.02149/) (2024)
- [Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks](https://mlsanity.com/paper/1808.03620/) (2018)
- [Ensemble Kalman Methods: A Mean Field Perspective](https://mlsanity.com/paper/2209.11371/) (2022)

## Built on by (most cited first, 19 of 19)

- [InverseBench: Benchmarking Plug-and-Play Diffusion Priors for Inverse Problems in Physical Sciences](https://mlsanity.com/paper/2503.11043/) (2025)
- [On the Guidance of Flow Matching](https://mlsanity.com/paper/2502.02150/) (2025)
- [DriftLite: Lightweight Drift Control for Inference-Time Scaling of Diffusion Models](https://mlsanity.com/paper/2509.21655/) (2025)
- [Split Gibbs Discrete Diffusion Posterior Sampling](https://mlsanity.com/paper/2503.01161/) (2025)
- [Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approach](https://mlsanity.com/paper/2506.03979/) (2025)
- [Free2Guide: Training-Free Text-to-Video Alignment using Image LVLM](https://mlsanity.com/paper/2411.17041/) (2024)
- [Nonlinear Assimilation via Score-based Sequential Langevin Sampling](https://mlsanity.com/paper/2411.13443/) (2024)
- [Blade: A Derivative-free Bayesian Inversion Method using Diffusion Priors](https://mlsanity.com/paper/2510.10968/) (2025)
- [Training-Free Bayesian Filtering with Generative Emulators](https://mlsanity.com/paper/2605.20028/) (2026)
- [STeP: A Framework for Solving Scientific Video Inverse Problems with Spatiotemporal Diffusion Priors](https://mlsanity.com/paper/2504.07549/) (2025)
- [EquiReg: Equivariance Regularized Diffusion for Inverse Problems](https://mlsanity.com/paper/2505.22973/) (2025)
- [Derivative-Free Diffusion Manifold-Constrained Gradient for Unified XAI](https://mlsanity.com/paper/2411.15265/) (2024)
- [Constrained Particle Seeking: Solving Diffusion Inverse Problems with Just Forward Passes](https://mlsanity.com/paper/2603.01837/) (2026)
- [Zeroth-Order Non-Log-Concave Sampling with Variance Reduction and Applications to Inverse Problems](https://mlsanity.com/paper/2605.30573/) (2026)
- [K-TRAIL: Simulator-Guided Generative Design of EM/RF Circuits](https://mlsanity.com/paper/2609.23183/) (2026)
- [Energy-Guided Generative Modeling for Low-Energy Molecular Structure Discovery](https://mlsanity.com/paper/2512.22597/) (2025)
- [KHMP: Frequency-Domain Kalman Refinement for High-Fidelity Human Motion Prediction](https://mlsanity.com/paper/2603.21327/) (2026)
- [Aligning One-Step Generative Models with Reward-Weighted Transport Distillation](https://mlsanity.com/paper/2609.30840/) (2026)
- [Robust Ensemble Guidance for Scientific Inverse Problems](https://mlsanity.com/paper/2610.05371/) (2026)

## Abstract (arXiv)

When solving inverse problems, one increasingly popular approach is to use pre-trained diffusion models as plug-and-play priors. This framework can accommodate different forward models without re-training while preserving the generative capability of diffusion models. Despite their success in many imaging inverse problems, most existing methods rely on privileged information such as derivative, pseudo-inverse, or full knowledge about the forward model. This reliance poses a substantial limitation that restricts their use in a wide range of problems where such information is unavailable, such as in many scientific applications. We propose Ensemble Kalman Diffusion Guidance (EnKG), a derivative-free approach that can solve inverse problems by only accessing forward model evaluations and a pre-trained diffusion model prior. We study the empirical effectiveness of EnKG across various inverse problems, including scientific settings such as inferring fluid flows and astronomical objects, which are highly non-linear inverse problems that often only permit black-box access to the forward model. We open-source our code at https://github.com/devzhk/enkg-pytorch.