# Dynamic Structured Illumination Microscopy with a Neural Space-time Model

arXiv: https://arxiv.org/abs/2206.01397 · HTML: https://mlsanity.com/paper/2206.01397/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 47 references that are in the map.
- First posted on arXiv 3 Jun 2022.

## Builds on (most cited first, 12 of 12)

- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [ACORN: Adaptive Coordinate Networks for Neural Scene Representation](https://mlsanity.com/paper/2105.02788/) (2021)
- [Structured illumination microscopy image reconstruction algorithm](https://mlsanity.com/paper/1602.06904/) (2016)
- [Structured illumination microscopy with unknown patterns and a statistical prior](https://mlsanity.com/paper/1611.00287/) (2016)
- [Computational structured illumination for high-content fluorescent and phase microscopy](https://mlsanity.com/paper/1901.00417/) (2018)

## Built on by (most cited first, 3 of 3)

- [HDNet: Physics-Inspired Neural Network for Flow Estimation based on Helmholtz Decomposition](https://mlsanity.com/paper/2406.08570/) (2024)
- [Recovering a Molecule's 3D Dynamics from Liquid-phase Electron Microscopy Movies](https://mlsanity.com/paper/2308.11927/) (2023)
- [Multi-modal deformable image registration using untrained neural networks](https://mlsanity.com/paper/2411.02672/) (2024)

## Abstract (arXiv)

Structured illumination microscopy (SIM) reconstructs a super-resolved image from multiple raw images captured with different illumination patterns; hence, acquisition speed is limited, making it unsuitable for dynamic scenes. We propose a new method, Speckle Flow SIM, that uses static patterned illumination with moving samples and models the sample motion during data capture in order to reconstruct the dynamic scene with super-resolution. Speckle Flow SIM relies on sample motion to capture a sequence of raw images. The spatio-temporal relationship of the dynamic scene is modeled using a neural space-time model with coordinate-based multi-layer perceptrons (MLPs), and the motion dynamics and the super-resolved scene are jointly recovered. We validate Speckle Flow SIM for coherent imaging in simulation and build a simple, inexpensive experimental setup with off-the-shelf components. We demonstrate that Speckle Flow SIM can reconstruct a dynamic scene with deformable motion and 1.88x the diffraction-limited resolution in experiment.