# Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning

arXiv: https://arxiv.org/abs/1901.11252 · HTML: https://mlsanity.com/paper/1901.11252/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jan 2019.

## Built on by (most cited first, 11 of 11)

- [Virtual staining of defocused autofluorescence images of unlabeled tissue using deep neural networks](https://mlsanity.com/paper/2207.02946/) (2022)
- [DiffuseIR:Diffusion Models For Isotropic Reconstruction of 3D Microscopic Images](https://mlsanity.com/paper/2306.12109/) (2023)
- [Neural network-based image reconstruction in swept-source optical coherence tomography using undersampled spectral data](https://mlsanity.com/paper/2103.03877/) (2021)
- [Diffractive Interconnects: All-Optical Permutation Operation Using Diffractive Networks](https://mlsanity.com/paper/2206.10152/) (2022)
- [Neural Network-Based Processing and Reconstruction of Compromised Biophotonic Image Data](https://mlsanity.com/paper/2403.14324/) (2024)
- [Advancing biological super-resolution microscopy through deep learning: a brief review](https://mlsanity.com/paper/2106.13064/) (2021)
- [Defocus Deblur Microscopy via Head-to-Tail Cross-scale Fusion](https://mlsanity.com/paper/2201.02876/) (2022)
- [Multi-scale Conditional Generative Modeling for Microscopic Image Restoration](https://mlsanity.com/paper/2407.05259/) (2024)
- [NeuRegenerate: A Framework for Visualizing Neurodegeneration](https://mlsanity.com/paper/2202.01115/) (2022)
- [Lightweight CycleGAN Models for Cross-Modality Image Transformation and Experimental Quality Assessment in Fluorescence Microscopy](https://mlsanity.com/paper/2510.15579/) (2025)
- [Physics-Informed Deep Learning Model for Cross-Modality Super-Resolution in Fluorescence Microscopy](https://mlsanity.com/paper/2607.21190/) (2026)

## Abstract (arXiv)

Three-dimensional (3D) fluorescence microscopy in general requires axial scanning to capture images of a sample at different planes. Here we demonstrate that a deep convolutional neural network can be trained to virtually refocus a 2D fluorescence image onto user-defined 3D surfaces within the sample volume. With this data-driven computational microscopy framework, we imaged the neuron activity of a Caenorhabditis elegans worm in 3D using a time-sequence of fluorescence images acquired at a single focal plane, digitally increasing the depth-of-field of the microscope by 20-fold without any axial scanning, additional hardware, or a trade-off of imaging resolution or speed. Furthermore, we demonstrate that this learning-based approach can correct for sample drift, tilt, and other image aberrations, all digitally performed after the acquisition of a single fluorescence image. This unique framework also cross-connects different imaging modalities to each other, enabling 3D refocusing of a single wide-field fluorescence image to match confocal microscopy images acquired at different sample planes. This deep learning-based 3D image refocusing method might be transformative for imaging and tracking of 3D biological samples, especially over extended periods of time, mitigating photo-toxicity, sample drift, aberration and defocusing related challenges associated with standard 3D fluorescence microscopy techniques.