# Generative modeling of living cells with SO(3)-equivariant implicit neural representations

arXiv: https://arxiv.org/abs/2304.08960 · HTML: https://mlsanity.com/paper/2304.08960/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 74 references that are in the map.
- First posted on arXiv 18 Apr 2023.

## Builds on (most cited first, 27 of 27)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Neural Fields in Visual Computing and Beyond](https://mlsanity.com/paper/2111.11426/) (2021)
- [Vector Neurons: A General Framework for SO(3)-Equivariant Networks](https://mlsanity.com/paper/2104.12229/) (2021)
- [A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups](https://mlsanity.com/paper/2104.09459/) (2021)
- [HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusion](https://mlsanity.com/paper/2303.17015/) (2023)
- [NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstruction](https://mlsanity.com/paper/2108.10991/) (2021)
- [CoIL: Coordinate-based Internal Learning for Imaging Inverse Problems](https://mlsanity.com/paper/2102.05181/) (2021)
- [Explicitly disentangling image content from translation and rotation with spatial-VAE](https://mlsanity.com/paper/1909.11663/) (2019)
- [Three Dimensional Fluorescence Microscopy Image Synthesis and Segmentation](https://mlsanity.com/paper/1801.07198/) (2018)
- [Frame Averaging for Equivariant Shape Space Learning](https://mlsanity.com/paper/2112.01741/) (2021)
- [Microscopy Cell Segmentation via Convolutional LSTM Networks](https://mlsanity.com/paper/1805.11247/) (2018)
- [EmbedTrack -- Simultaneous Cell Segmentation and Tracking Through Learning Offsets and Clustering Bandwidths](https://mlsanity.com/paper/2204.10713/) (2022)
- [GANs for Biological Image Synthesis](https://mlsanity.com/paper/1708.04692/) (2017)
- [Red blood cell image generation for data augmentation using Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1901.06219/) (2019)
- [Implicit Neural Representations for Generative Modeling of Living Cell Shapes](https://mlsanity.com/paper/2207.06283/) (2022)
- [Implicit Neural Representations for Modeling of Abdominal Aortic Aneurysm Progression](https://mlsanity.com/paper/2303.01069/) (2023)
- [Local Implicit Neural Representations for Multi-Sequence MRI Translation](https://mlsanity.com/paper/2302.01031/) (2023)
- [Generating Semi-Synthetic Validation Benchmarks for Embryomics](https://mlsanity.com/paper/1604.04906/) (2016)

## Built on by (most cited first, 4 of 4)

- [Vestibular schwannoma growth prediction from longitudinal MRI by time conditioned neural fields](https://mlsanity.com/paper/2404.02614/) (2024)
- [Steerable Anatomical Shape Synthesis with Implicit Neural Representations](https://mlsanity.com/paper/2504.03313/) (2025)
- [A Diffusion Model for Simulation Ready Coronary Anatomy with Morpho-skeletal Control](https://mlsanity.com/paper/2407.15631/) (2024)
- [GrowFields: Compositional 4D Neural Fields for Topology-Changing Plant Growth](https://mlsanity.com/paper/2607.03330/) (2026)

## Abstract (arXiv)

Data-driven cell tracking and segmentation methods in biomedical imaging require diverse and information-rich training data. In cases where the number of training samples is limited, synthetic computer-generated data sets can be used to improve these methods. This requires the synthesis of cell shapes as well as corresponding microscopy images using generative models. To synthesize realistic living cell shapes, the shape representation used by the generative model should be able to accurately represent fine details and changes in topology, which are common in cells. These requirements are not met by 3D voxel masks, which are restricted in resolution, and polygon meshes, which do not easily model processes like cell growth and mitosis. In this work, we propose to represent living cell shapes as level sets of signed distance functions (SDFs) which are estimated by neural networks. We optimize a fully-connected neural network to provide an implicit representation of the SDF value at any point in a 3D+time domain, conditioned on a learned latent code that is disentangled from the rotation of the cell shape. We demonstrate the effectiveness of this approach on cells that exhibit rapid deformations (Platynereis dumerilii), cells that grow and divide (C. elegans), and cells that have growing and branching filopodial protrusions (A549 human lung carcinoma cells). A quantitative evaluation using shape features and Dice similarity coefficients of real and synthetic cell shapes shows that our model can generate topologically plausible complex cell shapes in 3D+time with high similarity to real living cell shapes. Finally, we show how microscopy images of living cells that correspond to our generated cell shapes can be synthesized using an image-to-image model.