# Latent-space disentanglement with untrained generator networks for the isolation of different motion types in video data

arXiv: https://arxiv.org/abs/2205.10367 · HTML: https://mlsanity.com/paper/2205.10367/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 20 May 2022.

## Builds on (most cited first, 9 of 9)

- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [Variational Autoencoders and Nonlinear ICA: A Unifying Framework](https://mlsanity.com/paper/1907.04809/) (2019)
- [Deep Learning in Medical Image Registration: A Review](https://mlsanity.com/paper/1912.12318/) (2019)
- [Time-Dependent Deep Image Prior for Dynamic MRI](https://mlsanity.com/paper/1910.01684/) (2019)
- [Disentangling Identifiable Features from Noisy Data with Structured Nonlinear ICA](https://mlsanity.com/paper/2106.09620/) (2021)
- [Deformation corrected compressed sensing (DC-CS): a novel framework for accelerated dynamic MRI](https://mlsanity.com/paper/1405.7718/) (2014)
- [Generative Models for Low-Rank Video Representation and Reconstruction](https://mlsanity.com/paper/1902.11132/) (2019)

## Abstract (arXiv)

Isolating different types of motion in video data is a highly relevant problem in video analysis. Applications can be found, for example, in dynamic medical or biological imaging, where the analysis and further processing of the dynamics of interest is often complicated by additional, unwanted dynamics, such as motion of the measurement subject. In this work, it is empirically shown that a representation of video data via untrained generator networks, together with a specific technique for latent space disentanglement that uses minimal, one-dimensional information on some of the underlying dynamics, allows to efficiently isolate different, highly non-linear motion types. In particular, such a representation allows to freeze any selection of motion types, and to obtain accurate independent representations of other dynamics of interest. Obtaining such a representation does not require any pre-training on a training data set, i.e., all parameters of the generator network are learned directly from a single video.