# From Diffusion to Resolution: Leveraging 2D Diffusion Models for 3D Super-Resolution Task

arXiv: https://arxiv.org/abs/2411.16792 · HTML: https://mlsanity.com/paper/2411.16792/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 30 of its 47 references that are in the map.
- First posted on arXiv 25 Nov 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 30)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Real-Time Intermediate Flow Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2011.06294/) (2020)
- [Focal Frequency Loss for Image Reconstruction and Synthesis](https://mlsanity.com/paper/2012.12821/) (2020)
- [Deblurring by Realistic Blurring](https://mlsanity.com/paper/2004.01860/) (2020)
- [Image Restoration with Mean-Reverting Stochastic Differential Equations](https://mlsanity.com/paper/2301.11699/) (2023)
- [DiffusionSat: A Generative Foundation Model for Satellite Imagery](https://mlsanity.com/paper/2312.03606/) (2023)
- [IoU Loss for 2D/3D Object Detection](https://mlsanity.com/paper/1908.03851/) (2019)
- [FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation](https://mlsanity.com/paper/2012.08512/) (2020)
- [A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images](https://mlsanity.com/paper/2209.13167/) (2022)
- [Conversion Between CT and MRI Images Using Diffusion and Score-Matching Models](https://mlsanity.com/paper/2209.12104/) (2022)
- [Video Interpolation with Diffusion Models](https://mlsanity.com/paper/2404.01203/) (2024)
- [EDiffSR: An Efficient Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution](https://mlsanity.com/paper/2310.19288/) (2023)
- [Improving 3D Imaging with Pre-Trained Perpendicular 2D Diffusion Models](https://mlsanity.com/paper/2303.08440/) (2023)
- [Enhanced Quadratic Video Interpolation](https://mlsanity.com/paper/2009.04642/) (2020)
- [Continuous Dice Coefficient: a Method for Evaluating Probabilistic Segmentations](https://mlsanity.com/paper/1906.11031/) (2019)
- [Arbitrary-Scale Image Generation and Upsampling using Latent Diffusion Model and Implicit Neural Decoder](https://mlsanity.com/paper/2403.10255/) (2024)
- [AID: Attention Interpolation of Text-to-Image Diffusion](https://mlsanity.com/paper/2403.17924/) (2024)
- [Deep Learning for Isotropic Super-Resolution from Non-Isotropic 3D Electron Microscopy](https://mlsanity.com/paper/1706.03142/) (2017)
- [DiffuseIR:Diffusion Models For Isotropic Reconstruction of 3D Microscopic Images](https://mlsanity.com/paper/2306.12109/) (2023)
- [Reference-Free Isotropic 3D EM Reconstruction using Diffusion Models](https://mlsanity.com/paper/2308.01594/) (2023)
- [MicroDiffusion: Implicit Representation-Guided Diffusion for 3D Reconstruction from Limited 2D Microscopy Projections](https://mlsanity.com/paper/2403.10815/) (2024)
- [Super-resolution of biomedical volumes with 2D supervision](https://mlsanity.com/paper/2404.09425/) (2024)
- [Reference-free Axial Super-resolution of 3D Microscopy Images using Implicit Neural Representation with a 2D Diffusion Prior](https://mlsanity.com/paper/2408.08616/) (2024)

## Abstract (arXiv)

Diffusion models have recently emerged as a powerful technique in image generation, especially for image super-resolution tasks. While 2D diffusion models significantly enhance the resolution of individual images, existing diffusion-based methods for 3D volume super-resolution often struggle with structure discontinuities in axial direction and high sampling costs. In this work, we present a novel approach that leverages the 2D diffusion model and lateral continuity within the volume to enhance 3D volume electron microscopy (vEM) super-resolution. We first simulate lateral degradation with slices in the XY plane and train a 2D diffusion model to learn how to restore the degraded slices. The model is then applied slice-by-slice in the lateral direction of low-resolution volume, recovering slices while preserving inherent lateral continuity. Following this, a high-frequency-aware 3D super-resolution network is trained on the recovery lateral slice sequences to learn spatial feature transformation across slices. Finally, the network is applied to infer high-resolution volumes in the axial direction, enabling 3D super-resolution. We validate our approach through comprehensive evaluations, including image similarity assessments, resolution analysis, and performance on downstream tasks. Our results on two publicly available focused ion beam scanning electron microscopy (FIB-SEM) datasets demonstrate the robustness and practical applicability of our framework for 3D volume super-resolution.