# Local Conditional Neural Fields for Versatile and Generalizable Large-Scale Reconstructions in Computational Imaging

arXiv: https://arxiv.org/abs/2307.06207 · HTML: https://mlsanity.com/paper/2307.06207/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 52 references that are in the map.
- First posted on arXiv 12 Jul 2023.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Learning Continuous Image Representation with Local Implicit Image Function](https://mlsanity.com/paper/2012.09161/) (2020)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstruction](https://mlsanity.com/paper/2108.10991/) (2021)
- [Dynamic CT Reconstruction from Limited Views with Implicit Neural Representations and Parametric Motion Fields](https://mlsanity.com/paper/2104.11745/) (2021)
- [Phase recovery and holographic image reconstruction using deep learning in neural networks](https://mlsanity.com/paper/1705.04286/) (2017)
- [Recovery of Continuous 3D Refractive Index Maps from Discrete Intensity-Only Measurements using Neural Fields](https://mlsanity.com/paper/2112.00002/) (2021)
- [Lensless computational imaging through deep learning](https://mlsanity.com/paper/1702.08516/) (2017)
- [Reliable deep-learning-based phase imaging with uncertainty quantification](https://mlsanity.com/paper/1901.02038/) (2019)
- [FlatNet: Towards Photorealistic Scene Reconstruction from Lensless Measurements](https://mlsanity.com/paper/2010.15440/) (2020)
- [Deep Phase Decoder: Self-calibrating phase microscopy with an untrained deep neural network](https://mlsanity.com/paper/2001.09803/) (2020)
- [Deep speckle correlation: a deep learning approach towards scalable imaging through scattering media](https://mlsanity.com/paper/1806.04139/) (2018)
- [Experimental robustness of Fourier Ptychography phase retrieval algorithms](https://mlsanity.com/paper/1511.02986/) (2015)
- [Roadmap on Deep Learning for Microscopy](https://mlsanity.com/paper/2303.03793/) (2023)
- [A Memory-Efficient Dynamic Image Reconstruction Method using Neural Fields](https://mlsanity.com/paper/2205.05585/) (2022)
- [Adaptive 3D descattering with a dynamic synthesis network](https://mlsanity.com/paper/2107.00484/) (2021)
- [Deep-learning-augmented Computational Miniature Mesoscope](https://mlsanity.com/paper/2205.00123/) (2022)
- [Physics-informed neural networks for diffraction tomography](https://mlsanity.com/paper/2207.14230/) (2022)
- [High-resolution tomographic reconstruction of optical absorbance through scattering media using neural fields](https://mlsanity.com/paper/2304.01682/) (2023)

## Built on by (most cited first, 2 of 2)

- [On the use of deep learning for phase recovery](https://mlsanity.com/paper/2308.00942/) (2023)
- [ConVRT: Consistent Video Restoration Through Turbulence with Test-time Optimization of Neural Video Representations](https://mlsanity.com/paper/2312.04679/) (2023)

## Abstract (arXiv)

Deep learning has transformed computational imaging, but traditional pixel-based representations limit their ability to capture continuous, multiscale details of objects. Here we introduce a novel Local Conditional Neural Fields (LCNF) framework, leveraging a continuous implicit neural representation to address this limitation. LCNF enables flexible object representation and facilitates the reconstruction of multiscale information. We demonstrate the capabilities of LCNF in solving the highly ill-posed inverse problem in Fourier ptychographic microscopy (FPM) with multiplexed measurements, achieving robust, scalable, and generalizable large-scale phase retrieval. Unlike traditional neural fields frameworks, LCNF incorporates a local conditional representation that promotes model generalization, learning multiscale information, and efficient processing of large-scale imaging data. By combining an encoder and a decoder conditioned on a learned latent vector, LCNF achieves versatile continuous-domain super-resolution image reconstruction. We demonstrate accurate reconstruction of wide field-of-view, high-resolution phase images using only a few multiplexed measurements. LCNF robustly captures the continuous object priors and eliminates various phase artifacts, even when it is trained on imperfect datasets. The framework exhibits strong generalization, reconstructing diverse objects even with limited training data. Furthermore, LCNF can be trained on a physics simulator using natural images and successfully applied to experimental measurements on biological samples. Our results highlight the potential of LCNF for solving large-scale inverse problems in computational imaging, with broad applicability in various deep-learning-based techniques.