# Disentangling Space and Time in Video with Hierarchical Variational Auto-encoders

arXiv: https://arxiv.org/abs/1612.04440 · HTML: https://mlsanity.com/paper/1612.04440/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2016.

## Built on by (most cited first, 4 of 4)

- [Efficient Out-of-Distribution Detection Using Latent Space of β-VAE for Cyber-Physical Systems](https://mlsanity.com/paper/2108.11800/) (2021)
- [VDSM: Unsupervised Video Disentanglement with State-Space Modeling and Deep Mixtures of Experts](https://mlsanity.com/paper/2103.07292/) (2021)
- [Topographic VAEs learn Equivariant Capsules](https://mlsanity.com/paper/2109.01394/) (2021)
- [One Style is All you Need to Generate a Video](https://mlsanity.com/paper/2310.17835/) (2023)

## Abstract (arXiv)

There are many forms of feature information present in video data. Principle among them are object identity information which is largely static across multiple video frames, and object pose and style information which continuously transforms from frame to frame. Most existing models confound these two types of representation by mapping them to a shared feature space. In this paper we propose a probabilistic approach for learning separable representations of object identity and pose information using unsupervised video data. Our approach leverages a deep generative model with a factored prior distribution that encodes properties of temporal invariances in the hidden feature set. Learning is achieved via variational inference. We present results of learning identity and pose information on a dataset of moving characters as well as a dataset of rotating 3D objects. Our experimental results demonstrate our model's success in factoring its representation, and demonstrate that the model achieves improved performance in transfer learning tasks.