# Spatial-Temporal Pre-Training for Embryo Viability Prediction Using Time-Lapse Videos

arXiv: https://arxiv.org/abs/2506.17403 · HTML: https://mlsanity.com/paper/2506.17403/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 20 Jun 2025.

## Builds on (most cited first, 20 of 20)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Spatiotemporal Contrastive Video Representation Learning](https://mlsanity.com/paper/2008.03800/) (2020)
- [Debiased Contrastive Learning](https://mlsanity.com/paper/2007.00224/) (2020)
- [Temporal Cycle-Consistency Learning](https://mlsanity.com/paper/1904.07846/) (2019)
- [Representation Learning via Global Temporal Alignment and Cycle-Consistency](https://mlsanity.com/paper/2105.05217/) (2021)
- [Temperature Schedules for Self-Supervised Contrastive Methods on Long-Tail Data](https://mlsanity.com/paper/2303.13664/) (2023)
- [An Alternative Probabilistic Interpretation of the Huber Loss](https://mlsanity.com/paper/1911.02088/) (2019)
- [Video Contrastive Learning with Global Context](https://mlsanity.com/paper/2108.02722/) (2021)
- [Frame-wise Action Representations for Long Videos via Sequence Contrastive Learning](https://mlsanity.com/paper/2203.14957/) (2022)
- [Learning Vector Quantized Shape Code for Amodal Blastomere Instance Segmentation](https://mlsanity.com/paper/2012.00985/) (2020)
- [Automated Measurements of Key Morphological Features of Human Embryos for IVF](https://mlsanity.com/paper/2006.00067/) (2020)
- [Robust and generalizable embryo selection based on artificial intelligence and time-lapse image sequences](https://mlsanity.com/paper/2103.07262/) (2021)
- [Embryo staging with weakly-supervised region selection and dynamically-decoded predictions](https://mlsanity.com/paper/1904.04419/) (2019)

## Abstract (arXiv)

Automating embryo viability prediction for in vitro fertilization (IVF) is important but challenging due to the limited availability of labeled pregnancy outcome data, as only a small fraction of embryos are labeled after transfer. Self-supervised learning (SSL) can leverage both labeled and unlabeled data to improve prediction. However, existing SSL methods for videos are not directly applicable to embryo development videos due to two challenges: (1) embryo time-lapse videos contain hundreds of frames, requiring significant GPU memory for conventional SSL; (2) the dataset contains videos with varying lengths and many outlier frames, causing traditional video alignment methods to struggle with semantic misalignment. We propose Spatial-Temporal Pre-Training (STPT) to address these challenges. STPT includes two stages: spatial and temporal. In each stage, only one encoder is trained while the other is frozen, reducing memory demands. To handle temporal misalignment, STPT avoids frame-by-frame alignment across videos. The spatial stage learns from alignments within each video and its temporally consistent augmentations. The temporal stage then models relationships between video embeddings. Our method efficiently handles long videos and temporal variability. On 23,027 time-lapse videos (3,286 labeled), STPT achieves the highest AUC of 0.635 (95% CI: 0.632-0.638) compared to baselines, with limited computational resources.