# Deep Diffusion Image Prior for Efficient OOD Adaptation in 3D Inverse Problems

arXiv: https://arxiv.org/abs/2407.10641 · HTML: https://mlsanity.com/paper/2407.10641/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 46 of its 66 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 46)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation](https://mlsanity.com/paper/2305.16213/) (2023)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [Multi-Task Learning with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2009.09796/) (2020)
- [Denoising Diffusion Models for Plug-and-Play Image Restoration](https://mlsanity.com/paper/2305.08995/) (2023)
- [Score-based diffusion models for accelerated MRI](https://mlsanity.com/paper/2110.05243/) (2021)
- [Common Diffusion Noise Schedules and Sample Steps are Flawed](https://mlsanity.com/paper/2305.08891/) (2023)
- [Robust Compressed Sensing MRI with Deep Generative Priors](https://mlsanity.com/paper/2108.01368/) (2021)
- [Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models](https://mlsanity.com/paper/2305.10474/) (2023)
- [Learned Primal-dual Reconstruction](https://mlsanity.com/paper/1707.06474/) (2017)
- [Solving 3D Inverse Problems using Pre-trained 2D Diffusion Models](https://mlsanity.com/paper/2211.10655/) (2022)
- [Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problems](https://mlsanity.com/paper/2303.05754/) (2023)

## Built on by (most cited first, 21 of 21)

- [A Survey on Diffusion Models for Inverse Problems](https://mlsanity.com/paper/2410.00083/) (2024)
- [Diffusion models for inverse problems](https://mlsanity.com/paper/2508.01975/) (2025)
- [JointDiT: Enhancing RGB-Depth Joint Modeling with Diffusion Transformers](https://mlsanity.com/paper/2505.00482/) (2025)
- [Patch-Based Diffusion Models Beat Whole-Image Models for Mismatched Distribution Inverse Problems](https://mlsanity.com/paper/2410.11730/) (2024)
- [MRI Super-Resolution with Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2511.16854/) (2025)
- [PET Image Reconstruction Using Deep Diffusion Image Prior](https://mlsanity.com/paper/2507.15078/) (2025)
- [Moner: Motion Correction in Undersampled Radial MRI with Unsupervised Neural Representation](https://mlsanity.com/paper/2409.16921/) (2024)
- [Unsupervised Detection of Distribution Shift in Inverse Problems using Diffusion Models](https://mlsanity.com/paper/2505.11482/) (2025)
- [Improving 2D Diffusion Models for 3D Medical Imaging with Inter-Slice Consistent Stochasticity](https://mlsanity.com/paper/2602.04162/) (2026)
- [Mind the Detail: Uncovering Clinically Relevant Image Details in Accelerated MRI with Semantically Diverse Reconstructions](https://mlsanity.com/paper/2507.00670/) (2025)
- [Cross-Modal Guidance for Fast Diffusion-Based Computed Tomography](https://mlsanity.com/paper/2603.01253/) (2026)
- [Cross-Distribution Diffusion Priors-Driven Iterative Reconstruction for Sparse-View CT](https://mlsanity.com/paper/2509.13576/) (2025)
- [Equivariant Sampling for Improving Diffusion Model-based Image Restoration](https://mlsanity.com/paper/2511.09965/) (2025)
- [PMA-Diffusion: A Physics-guided Mask-Aware Diffusion Framework for TSE from Sparse Observations](https://mlsanity.com/paper/2512.06183/) (2025)
- [Local Patches Meet Global Context: Scalable 3D Diffusion Priors for Computed Tomography Reconstruction](https://mlsanity.com/paper/2512.18161/) (2025)
- [Regularizing INR with diffusion prior self-supervised 3D reconstruction of neutron computed tomography data](https://mlsanity.com/paper/2603.10947/) (2026)
- [Limited-Angle Tomography Reconstruction via Projector Guided 3D Diffusion](https://mlsanity.com/paper/2510.06516/) (2025)
- [Accelerating Stroke MRI with Diffusion Probabilistic Models through Large-Scale Pre-training and Target-Specific Fine-Tuning](https://mlsanity.com/paper/2603.13007/) (2026)
- [Toward a Foundation Plug-and-Play Prior for Computed Tomography Reconstruction via a Multimodal Diffusion Model](https://mlsanity.com/paper/2608.23190/) (2026)
- [Backward SDEs-based Diffusion for Physics-Constrained Generation](https://mlsanity.com/paper/2609.15702/) (2026)
- [STAR: Structure-aware Test-time Adaptation for diffusion-based light field Reconstruction](https://mlsanity.com/paper/2609.19747/) (2026)

## Abstract (arXiv)

Recent inverse problem solvers that leverage generative diffusion priors have garnered significant attention due to their exceptional quality. However, adaptation of the prior is necessary when there exists a discrepancy between the training and testing distributions. In this work, we propose deep diffusion image prior (DDIP), which generalizes the recent adaptation method of SCD by introducing a formal connection to the deep image prior. Under this framework, we propose an efficient adaptation method dubbed D3IP, specified for 3D measurements, which accelerates DDIP by orders of magnitude while achieving superior performance. D3IP enables seamless integration of 3D inverse solvers and thus leads to coherent 3D reconstruction. Moreover, we show that meta-learning techniques can also be applied to yield even better performance. We show that our method is capable of solving diverse 3D reconstructive tasks from the generative prior trained only with phantom images that are vastly different from the training set, opening up new opportunities of applying diffusion inverse solvers even when training with gold standard data is impossible. Code: https://github.com/HJ-harry/DDIP3D