# Physical model simulator-trained neural network for computational 3D phase imaging of multiple-scattering samples

arXiv: https://arxiv.org/abs/2103.15795 · HTML: https://mlsanity.com/paper/2103.15795/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 54 references that are in the map.
- First posted on arXiv 29 Mar 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Remote Sensing Image Scene Classification: Benchmark and State of the Art](https://mlsanity.com/paper/1703.00121/) (2017)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [Phase recovery and holographic image reconstruction using deep learning in neural networks](https://mlsanity.com/paper/1705.04286/) (2017)
- [Lensless computational imaging through deep learning](https://mlsanity.com/paper/1702.08516/) (2017)
- [PhaseStain: Digital staining of label-free quantitative phase microscopy images using deep learning](https://mlsanity.com/paper/1807.07701/) (2018)
- [Reliable deep-learning-based phase imaging with uncertainty quantification](https://mlsanity.com/paper/1901.02038/) (2019)
- [Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media](https://mlsanity.com/paper/1806.04139/) (2018)
- [Physics-based Learned Design: Optimized Coded-Illumination for Quantitative Phase Imaging](https://mlsanity.com/paper/1808.03571/) (2018)
- [SIMBA: Scalable Inversion in Optical Tomography using Deep Denoising Priors](https://mlsanity.com/paper/1911.13241/) (2019)
- [Diffraction tomography with a deep image prior](https://mlsanity.com/paper/1912.05330/) (2019)
- [Phase Imaging with Computational Specificity (PICS) for measuring dry mass changes in sub-cellular compartments](https://mlsanity.com/paper/2002.08361/) (2020)
- [High-Resolution Limited-Angle Phase Tomography of Dense Layered Objects Using Deep Neural Networks](https://mlsanity.com/paper/1812.07380/) (2018)
- [High-resolution 3D refractive index microscopy of multiple-scattering samples from intensity images](https://mlsanity.com/paper/1909.00023/) (2019)
- [On the interplay between physical and content priors in deep learning for computational imaging](https://mlsanity.com/paper/2004.06355/) (2020)
- [High-throughput intensity diffraction tomography with a computational microscope](https://mlsanity.com/paper/1801.09773/) (2018)
- [Efficient illumination angle self-calibration in Fourier ptychography](https://mlsanity.com/paper/1804.03299/) (2018)
- [Holographic particle localization under multiple scattering](https://mlsanity.com/paper/1807.11812/) (2018)
- [cellSTORM - Cost-effective Super-Resolution on a Cellphone using dSTORM](https://mlsanity.com/paper/1804.06244/) (2018)

## Built on by (most cited first, 3 of 3)

- [Recovery of Continuous 3D Refractive Index Maps from Discrete Intensity-Only Measurements using Neural Fields](https://mlsanity.com/paper/2112.00002/) (2021)
- [Physics-assisted Generative Adversarial Network for X-Ray Tomography](https://mlsanity.com/paper/2204.03703/) (2022)
- [Optical Diffraction Tomography based on 3D Physics-Inspired Neural Network (PINN)](https://mlsanity.com/paper/2206.05236/) (2022)

## Abstract (arXiv)

Recovering 3D phase features of complex, multiple-scattering biological samples traditionally sacrifices computational efficiency and processing time for physical model accuracy and reconstruction quality. This trade-off hinders the rapid analysis of living, dynamic biological samples that are often of greatest interest to biological research. Here, we overcome this bottleneck by combining annular intensity diffraction tomography (aIDT) with an approximant-guided deep learning framework. Using a novel physics model simulator-based learning strategy trained entirely on natural image datasets, we show our network can robustly reconstruct complex 3D biological samples of arbitrary size and structure. This approach highlights that large-scale multiple-scattering models can be leveraged in place of acquiring experimental datasets for achieving highly generalizable deep learning models. We devise a new model-based data normalization pre-processing procedure for homogenizing the sample contrast and achieving uniform prediction quality regardless of scattering strength. To achieve highly efficient training and prediction, we implement a lightweight 2D network structure that utilizes a multi-channel input for encoding the axial information. We demonstrate this framework's capabilities on experimental measurements of epithelial buccal cells and Caenorhabditis elegans worms. We highlight the robustness of this approach by evaluating dynamic samples on a living worm video, and we emphasize our approach's generalizability by recovering algae samples evaluated with different experimental setups. To assess the prediction quality, we develop a novel quantitative evaluation metric and show that our predictions are consistent with our experimental measurements and multiple-scattering physics.