# Self-supervised learning of hologram reconstruction using physics consistency

arXiv: https://arxiv.org/abs/2209.08288 · HTML: https://mlsanity.com/paper/2209.08288/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 110 references that are in the map.
- First posted on arXiv 17 Sep 2022.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Fourier Neural Operator for Parametric Partial Differential Equations](https://mlsanity.com/paper/2010.08895/) (2020)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [Self-supervised Learning is More Robust to Dataset Imbalance](https://mlsanity.com/paper/2110.05025/) (2021)
- [Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue](https://mlsanity.com/paper/1803.11293/) (2018)
- [Deep learning-based transformation of the H&E stain into special stains](https://mlsanity.com/paper/2008.08871/) (2020)
- [Phase recovery and holographic image reconstruction using deep learning in neural networks](https://mlsanity.com/paper/1705.04286/) (2017)
- [Probabilistic Noise2Void: Unsupervised Content-Aware Denoising](https://mlsanity.com/paper/1906.00651/) (2019)
- [PhaseStain: Digital staining of label-free quantitative phase microscopy images using deep learning](https://mlsanity.com/paper/1807.07701/) (2018)
- [Digital synthesis of histological stains using micro-structured and multiplexed virtual staining of label-free tissue](https://mlsanity.com/paper/2001.07267/) (2020)
- [Noise2Fast: Fast Self-Supervised Single Image Blind Denoising](https://mlsanity.com/paper/2108.10209/) (2021)
- [Low Photon Count Phase Retrieval Using Deep Learning](https://mlsanity.com/paper/1806.10029/) (2018)
- [Deep Phase Decoder: Self-calibrating phase microscopy with an untrained deep neural network](https://mlsanity.com/paper/2001.09803/) (2020)
- [AutoPhaseNN: Unsupervised Physics-aware Deep Learning of 3D Nanoscale Bragg Coherent Diffraction Imaging](https://mlsanity.com/paper/2109.14053/) (2021)
- [Fourier Imager Network (FIN): A deep neural network for hologram reconstruction with superior external generalization](https://mlsanity.com/paper/2204.10533/) (2022)
- [Cross-modality deep learning brings bright-field microscopy contrast to holography](https://mlsanity.com/paper/1811.07103/) (2018)
- [Learning to Synthesize: Robust Phase Retrieval at Low Photon counts](https://mlsanity.com/paper/1907.11713/) (2019)
- [PhaseGAN: A deep-learning phase-retrieval approach for unpaired datasets](https://mlsanity.com/paper/2011.08660/) (2020)
- [Quantitative Phase Imaging and Artificial Intelligence: A Review](https://mlsanity.com/paper/1806.03982/) (2018)
- [Phase Imaging with Computational Specificity (PICS) for measuring dry mass changes in sub-cellular compartments](https://mlsanity.com/paper/2002.08361/) (2020)
- [Single-shot autofocusing of microscopy images using deep learning](https://mlsanity.com/paper/2003.09585/) (2020)
- [Virtual staining of defocused autofluorescence images of unlabeled tissue using deep neural networks](https://mlsanity.com/paper/2207.02946/) (2022)
- [Deep Learning Microscopy](https://mlsanity.com/paper/1705.04709/) (2017)
- [Holographic image reconstruction with phase recovery and autofocusing using recurrent neural networks](https://mlsanity.com/paper/2102.12281/) (2021)
- [Deep learning enhanced mobile-phone microscopy](https://mlsanity.com/paper/1712.04139/) (2017)

## Built on by (most cited first, 14 of 14)

- [On the use of deep learning for phase recovery](https://mlsanity.com/paper/2308.00942/) (2023)
- [eFIN: Enhanced Fourier Imager Network for generalizable autofocusing and pixel super-resolution in holographic imaging](https://mlsanity.com/paper/2301.03162/) (2023)
- [A robust and scalable framework for hallucination detection in virtual tissue staining and digital pathology](https://mlsanity.com/paper/2404.18458/) (2024)
- [Neural Network-Based Processing and Reconstruction of Compromised Biophotonic Image Data](https://mlsanity.com/paper/2403.14324/) (2024)
- [All-optical complex field imaging using diffractive processors](https://mlsanity.com/paper/2401.16779/) (2024)
- [Multiplane Quantitative Phase Imaging Using a Wavelength-Multiplexed Diffractive Optical Processor](https://mlsanity.com/paper/2403.11035/) (2024)
- [Deep learning phase recovery: data-driven, physics-driven, or combining both?](https://mlsanity.com/paper/2404.01360/) (2024)
- [Single-shot reconstruction of three-dimensional morphology of biological cells in digital holographic microscopy using a physics-driven neural network](https://mlsanity.com/paper/2409.20013/) (2024)
- [OAH-Net: A Deep Neural Network for Hologram Reconstruction of Off-axis Digital Holographic Microscope](https://mlsanity.com/paper/2410.13592/) (2024)
- [Physics-informed generative real-time lens-free imaging](https://mlsanity.com/paper/2403.07786/) (2024)
- [Towards Robust and Generalizable Gerchberg Saxton based Physics Inspired Neural Networks for Computer Generated Holography: A Sensitivity Analysis Framework](https://mlsanity.com/paper/2505.00220/) (2025)
- [Self-supervised physics-informed generative networks for phase retrieval from a single X-ray hologram](https://mlsanity.com/paper/2508.15530/) (2025)
- [Generalizable Holographic Reconstruction via Amplitude-Only Diffusion Priors](https://mlsanity.com/paper/2509.12728/) (2025)
- [Restore What Matters: Lessons from Joint Restoration and Recognition](https://mlsanity.com/paper/2609.13791/) (2026)

## Abstract (arXiv)

The past decade has witnessed transformative applications of deep learning in various computational imaging, sensing and microscopy tasks. Due to the supervised learning schemes employed, these methods mostly depend on large-scale, diverse, and labeled training data. The acquisition and preparation of such training image datasets are often laborious and costly, also leading to biased estimation and limited generalization to new sample types. Here, we report a self-supervised learning model, termed GedankenNet, that eliminates the need for labeled or experimental training data, and demonstrate its effectiveness and superior generalization on hologram reconstruction tasks. Without prior knowledge about the sample types to be imaged, the self-supervised learning model was trained using a physics-consistency loss and artificial random images that are synthetically generated without any experiments or resemblance to real-world samples. After its self-supervised training, GedankenNet successfully generalized to experimental holograms of various unseen biological samples, reconstructing the phase and amplitude images of different types of objects using experimentally acquired test holograms. Without access to experimental data or knowledge of real samples of interest or their spatial features, GedankenNet's self-supervised learning achieved complex-valued image reconstructions that are consistent with the Maxwell's equations, and its output inference and object solutions accurately represent the wave propagation in free-space. GedankenNet framework also exhibits resilience to random, unknown perturbations in the physical forward model, including changes in the hologram distances, pixel size and illumination wavelength. This self-supervised learning of image reconstruction tasks creates new opportunities for various inverse problems in holography, microscopy and computational imaging fields.