# MicroDiffusion: Implicit Representation-Guided Diffusion for 3D Reconstruction from Limited 2D Microscopy Projections

arXiv: https://arxiv.org/abs/2403.10815 · HTML: https://mlsanity.com/paper/2403.10815/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 28 of its 49 references that are in the map.
- First posted on arXiv 16 Mar 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [Magic3D: High-Resolution Text-to-3D Content Creation](https://mlsanity.com/paper/2211.10440/) (2022)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [ImageBind: One Embedding Space To Bind Them All](https://mlsanity.com/paper/2305.05665/) (2023)
- [One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization](https://mlsanity.com/paper/2306.16928/) (2023)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors](https://mlsanity.com/paper/2306.17843/) (2023)
- [SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation](https://mlsanity.com/paper/2212.04493/) (2022)
- [3D Neural Field Generation using Triplane Diffusion](https://mlsanity.com/paper/2211.16677/) (2022)
- [Implicit Diffusion Models for Continuous Super-Resolution](https://mlsanity.com/paper/2303.16491/) (2023)
- [Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models](https://mlsanity.com/paper/2212.14704/) (2022)
- [NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstruction](https://mlsanity.com/paper/2108.10991/) (2021)
- [Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Data](https://mlsanity.com/paper/2306.07881/) (2023)
- [HoloDiffusion: Training a 3D Diffusion Model using 2D Images](https://mlsanity.com/paper/2303.16509/) (2023)
- [Reconstructing continuous distributions of 3D protein structure from cryo-EM images](https://mlsanity.com/paper/1909.05215/) (2019)
- [An Arbitrary Scale Super-Resolution Approach for 3D MR Images via Implicit Neural Representation](https://mlsanity.com/paper/2110.14476/) (2021)
- [Polynomial Implicit Neural Representations For Large Diverse Datasets](https://mlsanity.com/paper/2303.11424/) (2023)
- [SiReN: Sign-Aware Recommendation Using Graph Neural Networks](https://mlsanity.com/paper/2108.08735/) (2021)
- [FPM-INR: Fourier ptychographic microscopy image stack reconstruction using implicit neural representations](https://mlsanity.com/paper/2310.18529/) (2023)

## Built on by (most cited first, 6 of 6)

- [Where Do We Stand with Implicit Neural Representations? A Technical and Performance Survey](https://mlsanity.com/paper/2411.03688/) (2024)
- [GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstruction](https://mlsanity.com/paper/2512.25073/) (2025)
- [V2V3D: View-to-View Denoised 3D Reconstruction for Light-Field Microscopy](https://mlsanity.com/paper/2504.07853/) (2025)
- [I-INR: Iterative Implicit Neural Representations](https://mlsanity.com/paper/2504.17364/) (2025)
- [From Diffusion to Resolution: Leveraging 2D Diffusion Models for 3D Super-Resolution Task](https://mlsanity.com/paper/2411.16792/) (2024)
- [GeoTopoDiff: Learning Geometry--Topology Graph Priors through Boundary-Constrained Mixed Diffusion for Sparse-Slice 3D Porous Reconstruction](https://mlsanity.com/paper/2605.03764/) (2026)

## Abstract (arXiv)

Volumetric optical microscopy using non-diffracting beams enables rapid imaging of 3D volumes by projecting them axially to 2D images but lacks crucial depth information. Addressing this, we introduce MicroDiffusion, a pioneering tool facilitating high-quality, depth-resolved 3D volume reconstruction from limited 2D projections. While existing Implicit Neural Representation (INR) models often yield incomplete outputs and Denoising Diffusion Probabilistic Models (DDPM) excel at capturing details, our method integrates INR's structural coherence with DDPM's fine-detail enhancement capabilities. We pretrain an INR model to transform 2D axially-projected images into a preliminary 3D volume. This pretrained INR acts as a global prior guiding DDPM's generative process through a linear interpolation between INR outputs and noise inputs. This strategy enriches the diffusion process with structured 3D information, enhancing detail and reducing noise in localized 2D images. By conditioning the diffusion model on the closest 2D projection, MicroDiffusion substantially enhances fidelity in resulting 3D reconstructions, surpassing INR and standard DDPM outputs with unparalleled image quality and structural fidelity. Our code and dataset are available at https://github.com/UCSC-VLAA/MicroDiffusion.