# Learning Disentangled Representations of Video with Missing Data

arXiv: https://arxiv.org/abs/2006.13391 · HTML: https://mlsanity.com/paper/2006.13391/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jun 2020.

## Built on by (most cited first, 5 of 5)

- [Meta-Learning Dynamics Forecasting Using Task Inference](https://mlsanity.com/paper/2102.10271/) (2021)
- [CoRe-Sleep: A Multimodal Fusion Framework for Time Series Robust to Imperfect Modalities](https://mlsanity.com/paper/2304.06485/) (2023)
- [DEVIAS: Learning Disentangled Video Representations of Action and Scene](https://mlsanity.com/paper/2312.00826/) (2023)
- [Self-Supervised Decomposition, Disentanglement and Prediction of Video Sequences while Interpreting Dynamics: A Koopman Perspective](https://mlsanity.com/paper/2110.00547/) (2021)
- [Autoencoder-based General Purpose Representation Learning for Customer Embedding](https://mlsanity.com/paper/2402.18164/) (2024)

## Abstract (arXiv)

Missing data poses significant challenges while learning representations of video sequences. We present Disentangled Imputed Video autoEncoder (DIVE), a deep generative model that imputes and predicts future video frames in the presence of missing data. Specifically, DIVE introduces a missingness latent variable, disentangles the hidden video representations into static and dynamic appearance, pose, and missingness factors for each object. DIVE imputes each object's trajectory where data is missing. On a moving MNIST dataset with various missing scenarios, DIVE outperforms the state of the art baselines by a substantial margin. We also present comparisons for real-world MOTSChallenge pedestrian dataset, which demonstrates the practical value of our method in a more realistic setting. Our code and data can be found at https://github.com/Rose-STL-Lab/DIVE.