# Multi-StyleGAN: Towards Image-Based Simulation of Time-Lapse Live-Cell Microscopy

arXiv: https://arxiv.org/abs/2106.08285 · HTML: https://mlsanity.com/paper/2106.08285/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 15 Jun 2021.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [Which Training Methods for GANs do actually Converge?](https://mlsanity.com/paper/1801.04406/) (2018)
- [Image2StyleGAN++: How to Edit the Embedded Images?](https://mlsanity.com/paper/1911.11544/) (2019)
- [Kornia: an Open Source Differentiable Computer Vision Library for PyTorch](https://mlsanity.com/paper/1910.02190/) (2019)
- [A Note on the Inception Score](https://mlsanity.com/paper/1801.01973/) (2018)
- [A U-Net Based Discriminator for Generative Adversarial Networks](https://mlsanity.com/paper/2002.12655/) (2020)
- [Attention-Based Transformers for Instance Segmentation of Cells in Microstructures](https://mlsanity.com/paper/2011.09763/) (2020)
- [GANs for Biological Image Synthesis](https://mlsanity.com/paper/1708.04692/) (2017)
- [Red blood cell image generation for data augmentation using Conditional Generative Adversarial Networks](https://mlsanity.com/paper/1901.06219/) (2019)
- [Multiclass Yeast Segmentation in Microstructured Environments with Deep Learning](https://mlsanity.com/paper/2011.08062/) (2020)
- [Learning Generative Models of Tissue Organization with Supervised GANs](https://mlsanity.com/paper/2004.00140/) (2020)
- [Maximizing Information Gain for the Characterization of Biomolecular Circuits](https://mlsanity.com/paper/2101.02924/) (2021)
- [Generative Modeling with Conditional Autoencoders: Building an Integrated Cell](https://mlsanity.com/paper/1705.00092/) (2017)

## Built on by (most cited first, 4 of 4)

- [Histopathological Image Classification based on Self-Supervised Vision Transformer and Weak Labels](https://mlsanity.com/paper/2210.09021/) (2022)
- [Unveiling the Tapestry: the Interplay of Generalization and Forgetting in Continual Learning](https://mlsanity.com/paper/2211.11174/) (2022)
- [An Instance Segmentation Dataset of Yeast Cells in Microstructures](https://mlsanity.com/paper/2304.07597/) (2023)
- [A Perspective on Deep Vision Performance with Standard Image and Video Codecs](https://mlsanity.com/paper/2404.12330/) (2024)

## Abstract (arXiv)

Time-lapse fluorescent microscopy (TLFM) combined with predictive mathematical modelling is a powerful tool to study the inherently dynamic processes of life on the single-cell level. Such experiments are costly, complex and labour intensive. A complimentary approach and a step towards in silico experimentation, is to synthesise the imagery itself. Here, we propose Multi-StyleGAN as a descriptive approach to simulate time-lapse fluorescence microscopy imagery of living cells, based on a past experiment. This novel generative adversarial network synthesises a multi-domain sequence of consecutive timesteps. We showcase Multi-StyleGAN on imagery of multiple live yeast cells in microstructured environments and train on a dataset recorded in our laboratory. The simulation captures underlying biophysical factors and time dependencies, such as cell morphology, growth, physical interactions, as well as the intensity of a fluorescent reporter protein. An immediate application is to generate additional training and validation data for feature extraction algorithms or to aid and expedite development of advanced experimental techniques such as online monitoring or control of cells. Code and dataset is available at https://git.rwth-aachen.de/bcs/projects/tp/multi-stylegan.