# Recovering a Molecule's 3D Dynamics from Liquid-phase Electron Microscopy Movies

arXiv: https://arxiv.org/abs/2308.11927 · HTML: https://mlsanity.com/paper/2308.11927/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 47 references that are in the map.
- First posted on arXiv 23 Aug 2023.

## Builds on (most cited first, 27 of 27)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans](https://mlsanity.com/paper/2012.15838/) (2020)
- [Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video](https://mlsanity.com/paper/2012.12247/) (2020)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [Structured Inference Networks for Nonlinear State Space Models](https://mlsanity.com/paper/1609.09869/) (2016)
- [Deep Kalman Filters](https://mlsanity.com/paper/1511.05121/) (2015)
- [Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition](https://mlsanity.com/paper/1710.04340/) (2017)
- [Dynamical Variational Autoencoders: A Comprehensive Review](https://mlsanity.com/paper/2008.12595/) (2020)
- [Neural Rendering for Stereo 3D Reconstruction of Deformable Tissues in Robotic Surgery](https://mlsanity.com/paper/2206.15255/) (2022)
- [Disentangled Sequential Autoencoder](https://mlsanity.com/paper/1803.02991/) (2018)
- [Dynamic CT Reconstruction from Limited Views with Implicit Neural Representations and Parametric Motion Fields](https://mlsanity.com/paper/2104.11745/) (2021)
- [MedNeRF: Medical Neural Radiance Fields for Reconstructing 3D-aware CT-Projections from a Single X-ray](https://mlsanity.com/paper/2202.01020/) (2022)
- [Reconstructing continuous distributions of 3D protein structure from cryo-EM images](https://mlsanity.com/paper/1909.05215/) (2019)
- [Recovery of Continuous 3D Refractive Index Maps from Discrete Intensity-Only Measurements using Neural Fields](https://mlsanity.com/paper/2112.00002/) (2021)
- [CryoAI: Amortized Inference of Poses for Ab Initio Reconstruction of 3D Molecular Volumes from Real Cryo-EM Images](https://mlsanity.com/paper/2203.08138/) (2022)
- [Deep Generative Video Compression](https://mlsanity.com/paper/1810.02845/) (2018)
- [Amortized Inference for Heterogeneous Reconstruction in Cryo-EM](https://mlsanity.com/paper/2210.07387/) (2022)
- [Gravitationally Lensed Black Hole Emission Tomography](https://mlsanity.com/paper/2204.03715/) (2022)
- [DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction](https://mlsanity.com/paper/1908.03918/) (2019)
- [Dynamic Structured Illumination Microscopy with a Neural Space-time Model](https://mlsanity.com/paper/2206.01397/) (2022)
- [Improved Cryo-EM Pose Estimation and 3D Classification through Latent-Space Disentanglement](https://mlsanity.com/paper/2308.04956/) (2023)

## Built on by (most cited first, 2 of 2)

- [An Expectation-Maximization Algorithm for Training Clean Diffusion Models from Corrupted Observations](https://mlsanity.com/paper/2407.01014/) (2024)
- [AtomDiffuser: Time-Aware Degradation Modeling for Drift and Beam Damage in STEM Imaging](https://mlsanity.com/paper/2508.10359/) (2025)

## Abstract (arXiv)

The dynamics of biomolecules are crucial for our understanding of their functioning in living systems. However, current 3D imaging techniques, such as cryogenic electron microscopy (cryo-EM), require freezing the sample, which limits the observation of their conformational changes in real time. The innovative liquid-phase electron microscopy (liquid-phase EM) technique allows molecules to be placed in the native liquid environment, providing a unique opportunity to observe their dynamics. In this paper, we propose TEMPOR, a Temporal Electron MicroscoPy Object Reconstruction algorithm for liquid-phase EM that leverages an implicit neural representation (INR) and a dynamical variational auto-encoder (DVAE) to recover time series of molecular structures. We demonstrate its advantages in recovering different motion dynamics from two simulated datasets, 7bcq and Cas9. To our knowledge, our work is the first attempt to directly recover 3D structures of a temporally-varying particle from liquid-phase EM movies. It provides a promising new approach for studying molecules' 3D dynamics in structural biology.