mlsanityData to 14 Oct 2026

From Diffusion to Resolution: Leveraging 2D Diffusion Models for 3D Super-Resolution Task

arXiv 2411.16792 ↗First posted 25 Nov 2024cs.CVcited by 0

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Diffusion models have recently emerged as a powerful technique in image generation, especially for image super-resolution tasks. While 2D diffusion models significantly enhance the resolution of individual images, existing diffusion-based methods for 3D volume super-resolution often struggle with structure discontinuities in axial direction and high sampling costs. In this work, we present a novel approach that leverages the 2D diffusion model and lateral continuity within the volume to enhance 3D volume electron microscopy (vEM) super-resolution. We first simulate lateral degradation with slices in the XY plane and train a 2D diffusion model to learn how to restore the degraded slices. The model is then applied slice-by-slice in the lateral direction of low-resolution volume, recovering slices while preserving inherent lateral continuity. Following this, a high-frequency-aware 3D super-resolution network is trained on the recovery lateral slice sequences to learn spatial feature transformation across slices. Finally, the network is applied to infer high-resolution volumes in the axial direction, enabling 3D super-resolution. We validate our approach through comprehensive evaluations, including image similarity assessments, resolution analysis, and performance on downstream tasks. Our results on two publicly available focused ion beam scanning electron microscopy (FIB-SEM) datasets demonstrate the robustness and practical applicability of our framework for 3D volume super-resolution.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 30 of its 47 references that are in the map. First posted on arXiv 25 Nov 2024.

Builds on

30 of 47 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024Reference-free Axial Super-resolution of 3D Microscopy Images using Implicit Neural Representation with a 2D Diffusion Prior16 Aug
  2. 2024MicroDiffusion: Implicit Representation-Guided Diffusion for 3D Reconstruction from Limited 2D Microscopy Projections16 Mar
  3. 2024Arbitrary-Scale Image Generation and Upsampling using Latent Diffusion Model and Implicit Neural Decoder15 Mar
  4. 2023DiffusionSat: A Generative Foundation Model for Satellite Imagery6 Dec
  5. 2023EDiffSR: An Efficient Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution30 Oct
  6. 2023Reference-Free Isotropic 3D EM Reconstruction using Diffusion Models3 Aug
  7. 2023DiffuseIR:Diffusion Models For Isotropic Reconstruction of 3D Microscopic Images21 Jun

Other references · 23

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2022Classifier-Free Diffusion Guidancecited by 5,847
  5. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  6. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  7. 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 326
  8. 2019Depth-Aware Video Frame Interpolationcited by 234
  9. 2020Real-Time Intermediate Flow Estimation for Video Frame Interpolationcited by 213
  10. 2020Focal Frequency Loss for Image Reconstruction and Synthesiscited by 206
  11. 2020Deblurring by Realistic Blurringcited by 188
  12. 2023Image Restoration with Mean-Reverting Stochastic Differential Equationscited by 183
11 more
  1. 2019IoU Loss for 2D/3D Object Detectioncited by 128
  2. 2020FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolationcited by 98
  3. 2022A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Imagescited by 74
  4. 2022Conversion Between CT and MRI Images Using Diffusion and Score-Matching Modelscited by 69
  5. 2024Video Interpolation with Diffusion Modelscited by 68
  6. 2023Improving 3D Imaging with Pre-Trained Perpendicular 2D Diffusion Modelscited by 53
  7. 2020Enhanced Quadratic Video Interpolationcited by 48
  8. 2019Continuous Dice Coefficient: a Method for Evaluating Probabilistic Segmentationscited by 25
  9. 2024AID: Attention Interpolation of Text-to-Image Diffusioncited by 25
  10. 2017Deep Learning for Isotropic Super-Resolution from Non-Isotropic 3D Electron Microscopycited by 12
  11. 2024Super-resolution of biomedical volumes with 2D supervisioncited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SkelEM: Training-Signal Decoupling of Skeleton and Diffusion for Self-supervised Axial Super-Resolution in Volume Microscopy
  2. 2025EMGauss: Continuous Slice-to-3D Reconstruction via Dynamic Gaussian Modeling in Volume Electron Microscopy
  3. 2025Bridging Diffusion Models and 3D Representations: A 3D Consistent Super-Resolution Framework
  4. 2024Zoomed In, Diffused Out: Towards Local Degradation-Aware Multi-Diffusion for Extreme Image Super-Resolution
  5. 2026TOC-SR: Task-Optimal Compact diffusion for Image Super Resolution
  6. 2025Accelerating Diffusion-based Super-Resolution with Dynamic Time-Spatial Sampling
  7. 2023SinSR: Diffusion-Based Image Super-Resolution in a Single Step
  8. 2023Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution