mlsanityData to 14 Oct 2026

A Diffusion Model Predicts 3D Shapes from 2D Microscopy Images

arXiv 2208.14125 ↗First posted 30 Aug 2022cs.CV cs.AI cs.LGcited by 24

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Diffusion models are a special type of generative model, capable of synthesising new data from a learnt distribution. We introduce DISPR, a diffusion-based model for solving the inverse problem of three-dimensional (3D) cell shape prediction from two-dimensional (2D) single cell microscopy images. Using the 2D microscopy image as a prior, DISPR is conditioned to predict realistic 3D shape reconstructions. To showcase the applicability of DISPR as a data augmentation tool in a feature-based single cell classification task, we extract morphological features from the red blood cells grouped into six highly imbalanced classes. Adding features from the DISPR predictions to the three minority classes improved the macro F1 score from F1macro = 55.2 ± 4.6% to F1macro = 72.2 ± 4.9%. We thus demonstrate that diffusion models can be successfully applied to inverse biomedical problems, and that they learn to reconstruct 3D shapes with realistic morphological features from 2D microscopy images.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 16 of its 33 references that are in the map. First posted on arXiv 30 Aug 2022.

Builds on

16 of 33 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Unsupervised Medical Image Translation with Adversarial Diffusion Models17 Jul
  2. 2022Diffusion Models for Medical Anomaly Detection8 Mar
  3. 2021Diffusion Models for Implicit Image Segmentation Ensembles6 Dec

Other references · 13

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  8. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  9. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
  10. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
  11. 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
  12. 2022Diffusion Deformable Model for 4D Temporal Medical Image Generationcited by 65
1 more
  1. 2022Capturing Shape Information with Multi-Scale Topological Loss Terms for 3D Reconstructioncited by 14

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 6; 2024: 9; 2025: 4.

24 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  2. 2022Novel View Synthesis with Diffusion Modelscited by 300
  3. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Surveycited by 273
  4. 2023DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animationcited by 136
  5. 2022Solving 3D Inverse Problems using Pre-trained 2D Diffusion Modelscited by 132
  6. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 73
  7. 2024Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theorycited by 63
  8. 2023Diffusion Models in Bioinformatics: A New Wave of Deep Learning Revolution in Actioncited by 15
  9. 2024A Practical Guide to Sample-based Statistical Distances for Evaluating Generative Models in Sciencecited by 14
  10. 2023SADIR: Shape-Aware Diffusion Models for 3D Image Reconstructioncited by 12
  11. 2023Euler Characteristic Transform Based Topological Loss for Reconstructing 3D Images from Single 2D Slicescited by 6
  12. 2024CT Synthesis with Conditional Diffusion Models for Abdominal Lymph Node Segmentationcited by 5
12 more
  1. 2025PowerGrow: Feasible Co-Growth of Structures and Dynamics for Power Grid Synthesiscited by 5
  2. 2023A Unified Framework for Generative Data Augmentation: A Comprehensive Surveycited by 4
  3. 2025From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstructioncited by 4
  4. 2024Volumetric Conditioning Module to Control Pretrained Diffusion Models for 3D Medical Imagescited by 3
  5. 2022MMV_Im2Im: An Open Source Microscopy Machine Vision Toolbox for Image-to-Image Transformationcited by 2
  6. 2024X-ray2CTPA: Leveraging Diffusion Models to Enhance Pulmonary Embolism Classificationcited by 2
  7. 2024Mask-guided cross-image attention for zero-shot in-silico histopathologic image generation with a diffusion modelcited by 2
  8. 2023Self-supervised Learning for Enhancing Geometrical Modeling in 3D-Aware Generative Adversarial Networkcited by 1
  9. 2024QID2: An Image-Conditioned Diffusion Model for Q-space Up-sampling of DWI Datacited by 1
  10. 2024Diffusion-driven lensless fiber endomicroscopic quantitative phase imaging towards digital pathologycited by 0
  11. 2025Reconstruction of three-dimensional shapes of normal and disease-related erythrocytes from partial observations using multi-fidelity neural networkscited by 0
  12. 2025TRIDENT: A Trimodal Cascade Generative Framework for Drug and RNA-Conditioned Cellular Morphology Synthesiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Cascaded Diffusion Models for 2D and 3D Microscopy Image Synthesis to Enhance Cell Segmentation
  2. 2023Denoising Diffusion Probabilistic Models for Generation of Realistic Fully-Annotated Microscopy Image Data Sets
  3. 2025Diffusion-Based Synthetic Brightfield Microscopy Images for Enhanced Single Cell Detection
  4. 2025Revealing Subtle Phenotypes in Small Microscopy Datasets Using Latent Diffusion Models
  5. 2023Class-Guided Image-to-Image Diffusion: Cell Painting from Brightfield Images with Class Labels
  6. 2025Conditional Diffusion Models are Medical Image Classifiers that Provide Explainability and Uncertainty for Free
  7. 2023DiffuseIR:Diffusion Models For Isotropic Reconstruction of 3D Microscopic Images
  8. 2025Reducing Domain Gap with Diffusion-Based Domain Adaptation for Cell Counting