# Revealing Microscopic Objects in Fluorescence Live Imaging by Video-to-video Translation Based on A Spatial-temporal Generative Adversarial Network

arXiv: https://arxiv.org/abs/2502.16342 · HTML: https://mlsanity.com/paper/2502.16342/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 58 references that are in the map.
- First posted on arXiv 22 Feb 2025.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Generative Adversarial Network in Medical Imaging: A Review](https://mlsanity.com/paper/1809.07294/) (2018)
- [Coupled Generative Adversarial Networks](https://mlsanity.com/paper/1606.07536/) (2016)
- [Translating and Segmenting Multimodal Medical Volumes with Cycle- and Shape-Consistency Generative Adversarial Network](https://mlsanity.com/paper/1802.09655/) (2018)
- [Mocycle-GAN: Unpaired Video-to-Video Translation](https://mlsanity.com/paper/1908.09514/) (2019)
- [CoMusion: Towards Consistent Stochastic Human Motion Prediction via Motion Diffusion](https://mlsanity.com/paper/2305.12554/) (2023)
- [Elastica: A compliant mechanics environment for soft robotic control](https://mlsanity.com/paper/2009.08422/) (2020)
- [GANs for Biological Image Synthesis](https://mlsanity.com/paper/1708.04692/) (2017)
- [Learning Degradation-Independent Representations for Camera ISP Pipelines](https://mlsanity.com/paper/2307.00761/) (2023)
- [Deep Multi-modality Soft-decoding of Very Low Bit-rate Face Videos](https://mlsanity.com/paper/2008.01652/) (2020)
- [Functional Neural Networks for Parametric Image Restoration Problems](https://mlsanity.com/paper/2111.00361/) (2021)
- [Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learning](https://mlsanity.com/paper/2404.14607/) (2024)
- [Data Acquisition and Preparation for Dual-reference Deep Learning of Image Super-Resolution](https://mlsanity.com/paper/2108.02348/) (2021)
- [Revealing the Power of Masked Autoencoders in Traffic Forecasting](https://mlsanity.com/paper/2309.15169/) (2023)
- [Dynamic Graph Node Classification via Time Augmentation](https://mlsanity.com/paper/2212.03449/) (2022)
- [Towards Accurate Human Motion Prediction via Iterative Refinement](https://mlsanity.com/paper/2305.04443/) (2023)
- [MOOSS: Mask-Enhanced Temporal Contrastive Learning for Smooth State Evolution in Visual Reinforcement Learning](https://mlsanity.com/paper/2409.02714/) (2024)
- [Multi-Object Portion Tracking in 4D Fluorescence Microscopy Imagery with Deep Feature Maps](https://mlsanity.com/paper/1911.11808/) (2019)
- [Self-Supervised Face Image Restoration with a One-Shot Reference](https://mlsanity.com/paper/2203.03005/) (2022)
- [Bandit Learning to Rank with Position-Based Click Models: Personalized and Equal Treatments](https://mlsanity.com/paper/2311.04528/) (2023)
- [Automated Quantification of White Blood Cells in Light Microscopic Images of Injured Skeletal Muscle](https://mlsanity.com/paper/2409.06722/) (2024)

## Abstract (arXiv)

In spite of being a valuable tool to simultaneously visualize multiple types of subcellular structures using spectrally distinct fluorescent labels, a standard fluoresce microscope is only able to identify a few microscopic objects; such a limit is largely imposed by the number of fluorescent labels available to the sample. In order to simultaneously visualize more objects, in this paper, we propose to use video-to-video translation that mimics the development process of microscopic objects. In essence, we use a microscopy video-to-video translation framework namely Spatial-temporal Generative Adversarial Network (STGAN) to reveal the spatial and temporal relationships between the microscopic objects, after which a microscopy video of one object can be translated to another object in a different domain. The experimental results confirm that the proposed STGAN is effective in microscopy video-to-video translation that mitigates the spectral conflicts caused by the limited fluorescent labels, allowing multiple microscopic objects be simultaneously visualized.