# Improving 3D Imaging with Pre-Trained Perpendicular 2D Diffusion Models

arXiv: https://arxiv.org/abs/2303.08440 · HTML: https://mlsanity.com/paper/2303.08440/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 36 of its 50 references that are in the map.
- First posted on arXiv 15 Mar 2023.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 30 of 36)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Make-A-Video: Text-to-Video Generation without Text-Video Data](https://mlsanity.com/paper/2209.14792/) (2022)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Solving Inverse Problems in Medical Imaging with Score-Based Generative Models](https://mlsanity.com/paper/2111.08005/) (2021)
- [DiGress: Discrete Denoising diffusion for graph generation](https://mlsanity.com/paper/2209.14734/) (2022)
- [FastNeRF: High-Fidelity Neural Rendering at 200FPS](https://mlsanity.com/paper/2103.10380/) (2021)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speech](https://mlsanity.com/paper/2105.06337/) (2021)
- [Deep Convolutional Neural Network for Inverse Problems in Imaging](https://mlsanity.com/paper/1611.03679/) (2016)
- [Score-based diffusion models for accelerated MRI](https://mlsanity.com/paper/2110.05243/) (2021)
- [3D Neural Field Generation using Triplane Diffusion](https://mlsanity.com/paper/2211.16677/) (2022)
- [DiffRF: Rendering-Guided 3D Radiance Field Diffusion](https://mlsanity.com/paper/2212.01206/) (2022)
- [Dreamix: Video Diffusion Models are General Video Editors](https://mlsanity.com/paper/2302.01329/) (2023)
- [Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding](https://mlsanity.com/paper/2211.06956/) (2022)
- [FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesis](https://mlsanity.com/paper/2204.09934/) (2022)
- [Solving 3D Inverse Problems using Pre-trained 2D Diffusion Models](https://mlsanity.com/paper/2211.10655/) (2022)

## Built on by (most cited first, 30 of 53)

- [Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency](https://mlsanity.com/paper/2307.08123/) (2023)
- [Prompt-tuning latent diffusion models for inverse problems](https://mlsanity.com/paper/2310.01110/) (2023)
- [R2-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction](https://mlsanity.com/paper/2405.20693/) (2024)
- [Make-A-Volume: Leveraging Latent Diffusion Models for Cross-Modality 3D Brain MRI Synthesis](https://mlsanity.com/paper/2307.10094/) (2023)
- [Regularization by Texts for Latent Diffusion Inverse Solvers](https://mlsanity.com/paper/2311.15658/) (2023)
- [Deep Diffusion Image Prior for Efficient OOD Adaptation in 3D Inverse Problems](https://mlsanity.com/paper/2407.10641/) (2024)
- [Learning Image Priors through Patch-based Diffusion Models for Solving Inverse Problems](https://mlsanity.com/paper/2406.02462/) (2024)
- [Diffusion models for inverse problems](https://mlsanity.com/paper/2508.01975/) (2025)
- [Deep Generative Models for 3D Medical Image Synthesis](https://mlsanity.com/paper/2410.17664/) (2024)
- [2.5D Multi-view Averaging Diffusion Model for 3D Medical Image Translation: Application to Low-count PET Reconstruction with CT-less Attenuation Correction](https://mlsanity.com/paper/2406.08374/) (2024)
- [DiffusionBlend: Learning 3D Image Prior through Position-aware Diffusion Score Blending for 3D Computed Tomography Reconstruction](https://mlsanity.com/paper/2406.10211/) (2024)
- [DDPET-3D: Dose-aware Diffusion Model for 3D Ultra Low-dose PET Imaging](https://mlsanity.com/paper/2311.04248/) (2023)
- [A Survey of Emerging Applications of Diffusion Probabilistic Models in MRI](https://mlsanity.com/paper/2311.11383/) (2023)
- [Cascaded Multi-path Shortcut Diffusion Model for Medical Image Translation](https://mlsanity.com/paper/2405.12223/) (2024)
- [DiffuX2CT: Diffusion Learning to Reconstruct CT Images from Biplanar X-Rays](https://mlsanity.com/paper/2407.13545/) (2024)
- [Likelihood-Scheduled Score-Based Generative Modeling for Fully 3D PET Image Reconstruction](https://mlsanity.com/paper/2412.04339/) (2024)
- [DIFR3CT: Latent Diffusion for Probabilistic 3D CT Reconstruction from Few Planar X-Rays](https://mlsanity.com/paper/2408.15118/) (2024)
- [Regression is all you need for medical image translation](https://mlsanity.com/paper/2505.02048/) (2025)
- [X-LRM: X-ray Large Reconstruction Model for Extremely Sparse-View Computed Tomography Recovery in One Second](https://mlsanity.com/paper/2503.06382/) (2025)
- [X2-Gaussian: 4D Radiative Gaussian Splatting for Continuous-time Tomographic Reconstruction](https://mlsanity.com/paper/2503.21779/) (2025)
- [X-GRM: Large Gaussian Reconstruction Model for Sparse-view X-rays to Computed Tomography](https://mlsanity.com/paper/2505.15235/) (2025)
- [Two-and-a-half Order Score-based Model for Solving 3D Ill-posed Inverse Problems](https://mlsanity.com/paper/2308.08511/) (2023)
- [X-Diffusion: Generating Detailed 3D MRI Volumes From a Single Image Using Cross-Sectional Diffusion Models](https://mlsanity.com/paper/2404.19604/) (2024)
- [Dose-aware Diffusion Model for 3D PET Image Denoising: Multi-institutional Validation with Reader Study and Real Low-dose Data](https://mlsanity.com/paper/2405.12996/) (2024)
- [Physics-Inspired Generative Models in Medical Imaging: A Review](https://mlsanity.com/paper/2407.10856/) (2024)
- [GR-Gaussian: Graph-Based Radiative Gaussian Splatting for Sparse-View CT Reconstruction](https://mlsanity.com/paper/2508.02408/) (2025)
- [Blaze3DM: Marry Triplane Representation with Diffusion for 3D Medical Inverse Problem Solving](https://mlsanity.com/paper/2405.15241/) (2024)
- [Data-iterative Optimization Score Model for Stable Ultra-Sparse-View CT Reconstruction](https://mlsanity.com/paper/2308.14437/) (2023)
- [Super-resolution of biomedical volumes with 2D supervision](https://mlsanity.com/paper/2404.09425/) (2024)
- [Reference-free Axial Super-resolution of 3D Microscopy Images using Implicit Neural Representation with a 2D Diffusion Prior](https://mlsanity.com/paper/2408.08616/) (2024)

## Abstract (arXiv)

Diffusion models have become a popular approach for image generation and reconstruction due to their numerous advantages. However, most diffusion-based inverse problem-solving methods only deal with 2D images, and even recently published 3D methods do not fully exploit the 3D distribution prior. To address this, we propose a novel approach using two perpendicular pre-trained 2D diffusion models to solve the 3D inverse problem. By modeling the 3D data distribution as a product of 2D distributions sliced in different directions, our method effectively addresses the curse of dimensionality. Our experimental results demonstrate that our method is highly effective for 3D medical image reconstruction tasks, including MRI Z-axis super-resolution, compressed sensing MRI, and sparse-view CT. Our method can generate high-quality voxel volumes suitable for medical applications.