# A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images

arXiv: https://arxiv.org/abs/2208.14125 · HTML: https://mlsanity.com/paper/2208.14125/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 30 Aug 2022.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 16 of 16)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [Diffusion Models for Implicit Image Segmentation Ensembles](https://mlsanity.com/paper/2112.03145/) (2021)
- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [Diffusion Deformable Model for 4D Temporal Medical Image Generation](https://mlsanity.com/paper/2206.13295/) (2022)
- [Capturing Shape Information with Multi-Scale Topological Loss Terms for 3D Reconstruction](https://mlsanity.com/paper/2203.01703/) (2022)

## Built on by (most cited first, 24 of 24)

- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Novel View Synthesis with Diffusion Models](https://mlsanity.com/paper/2210.04628/) (2022)
- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation](https://mlsanity.com/paper/2301.03786/) (2023)
- [Solving 3D Inverse Problems using Pre-trained 2D Diffusion Models](https://mlsanity.com/paper/2211.10655/) (2022)
- [An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization](https://mlsanity.com/paper/2404.07771/) (2024)
- [Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory](https://mlsanity.com/paper/2403.11968/) (2024)
- [Diffusion Models in Bioinformatics: A New Wave of Deep Learning Revolution in Action](https://mlsanity.com/paper/2302.10907/) (2023)
- [A Practical Guide to Sample-based Statistical Distances for Evaluating Generative Models in Science](https://mlsanity.com/paper/2403.12636/) (2024)
- [SADIR: Shape-Aware Diffusion Models for 3D Image Reconstruction](https://mlsanity.com/paper/2309.03335/) (2023)
- [Euler Characteristic Transform Based Topological Loss for Reconstructing 3D Images from Single 2D Slices](https://mlsanity.com/paper/2303.05286/) (2023)
- [CT Synthesis with Conditional Diffusion Models for Abdominal Lymph Node Segmentation](https://mlsanity.com/paper/2403.17770/) (2024)
- [PowerGrow: Feasible Co-Growth of Structures and Dynamics for Power Grid Synthesis](https://mlsanity.com/paper/2509.12212/) (2025)
- [A Unified Framework for Generative Data Augmentation: A Comprehensive Survey](https://mlsanity.com/paper/2310.00277/) (2023)
- [From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstruction](https://mlsanity.com/paper/2505.03599/) (2025)
- [Volumetric Conditioning Module to Control Pretrained Diffusion Models for 3D Medical Images](https://mlsanity.com/paper/2410.21826/) (2024)
- [MMV_Im2Im: An Open Source Microscopy Machine Vision Toolbox for Image-to-Image Transformation](https://mlsanity.com/paper/2209.02498/) (2022)
- [X-ray2CTPA: Leveraging Diffusion Models to Enhance Pulmonary Embolism Classification](https://mlsanity.com/paper/2406.16109/) (2024)
- [Mask-guided cross-image attention for zero-shot in-silico histopathologic image generation with a diffusion model](https://mlsanity.com/paper/2407.11664/) (2024)
- [Self-supervised Learning for Enhancing Geometrical Modeling in 3D-Aware Generative Adversarial Network](https://mlsanity.com/paper/2312.11856/) (2023)
- [QID2: An Image-Conditioned Diffusion Model for Q-space Up-sampling of DWI Data](https://mlsanity.com/paper/2409.02309/) (2024)
- [Diffusion-driven lensless fiber endomicroscopic quantitative phase imaging towards digital pathology](https://mlsanity.com/paper/2407.18456/) (2024)
- [Reconstruction of three-dimensional shapes of normal and disease-related erythrocytes from partial observations using multi-fidelity neural networks](https://mlsanity.com/paper/2511.14962/) (2025)
- [TRIDENT: A Trimodal Cascade Generative Framework for Drug and RNA-Conditioned Cellular Morphology Synthesis](https://mlsanity.com/paper/2511.18287/) (2025)

## Abstract (arXiv)

Diffusion models are a special type of generative model, capable of synthesising new data from a learnt distribution. We introduce DISPR, a diffusion-based model for solving the inverse problem of three-dimensional (3D) cell shape prediction from two-dimensional (2D) single cell microscopy images. Using the 2D microscopy image as a prior, DISPR is conditioned to predict realistic 3D shape reconstructions. To showcase the applicability of DISPR as a data augmentation tool in a feature-based single cell classification task, we extract morphological features from the red blood cells grouped into six highly imbalanced classes. Adding features from the DISPR predictions to the three minority classes improved the macro F1 score from F1macro = 55.2 ± 4.6% to F1macro = 72.2 ± 4.9%. We thus demonstrate that diffusion models can be successfully applied to inverse biomedical problems, and that they learn to reconstruct 3D shapes with realistic morphological features from 2D microscopy images.