# Sequential Disentanglement by Extracting Static Information From A Single Sequence Element

arXiv: https://arxiv.org/abs/2406.18131 · HTML: https://mlsanity.com/paper/2406.18131/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 50 references that are in the map.
- First posted on arXiv 26 Jun 2024.

## Builds on (most cited first, 27 of 27)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [A Recurrent Latent Variable Model for Sequential Data](https://mlsanity.com/paper/1506.02216/) (2015)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [Variational Inference of Disentangled Latent Concepts from Unlabeled Observations](https://mlsanity.com/paper/1711.00848/) (2017)
- [Variational Lossy Autoencoder](https://mlsanity.com/paper/1611.02731/) (2016)
- [GP-VAE: Deep Probabilistic Time Series Imputation](https://mlsanity.com/paper/1907.04155/) (2019)
- [A Good Image Generator Is What You Need for High-Resolution Video Synthesis](https://mlsanity.com/paper/2104.15069/) (2021)
- [Unsupervised Learning of Disentangled Representations from Video](https://mlsanity.com/paper/1705.10915/) (2017)
- [Dynamical Variational Autoencoders: A Comprehensive Review](https://mlsanity.com/paper/2008.12595/) (2020)
- [Learning to Decompose and Disentangle Representations for Video Prediction](https://mlsanity.com/paper/1806.04166/) (2018)
- [Disentangled Sequential Autoencoder](https://mlsanity.com/paper/1803.02991/) (2018)
- [Multi-Level Variational Autoencoder: Learning Disentangled Representations from Grouped Observations](https://mlsanity.com/paper/1705.08841/) (2017)
- [Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data](https://mlsanity.com/paper/1709.07902/) (2017)
- [Lipschitz Recurrent Neural Networks](https://mlsanity.com/paper/2006.12070/) (2020)
- [S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation](https://mlsanity.com/paper/2005.11437/) (2020)
- [Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEs](https://mlsanity.com/paper/2310.02619/) (2023)
- [Contrastively Disentangled Sequential Variational Autoencoder](https://mlsanity.com/paper/2110.12091/) (2021)
- [Multifactor Sequential Disentanglement via Structured Koopman Autoencoders](https://mlsanity.com/paper/2303.17264/) (2023)
- [Disentangled Recurrent Wasserstein Autoencoder](https://mlsanity.com/paper/2101.07496/) (2021)
- [Decoupling Local and Global Representations of Time Series](https://mlsanity.com/paper/2202.02262/) (2022)
- [An Operator Theoretic Approach for Analyzing Sequence Neural Networks](https://mlsanity.com/paper/2102.07824/) (2021)
- [Sample and Predict Your Latent: Modality-free Sequential Disentanglement via Contrastive Estimation](https://mlsanity.com/paper/2305.15924/) (2023)
- [The Utility of Clustering in Prediction Tasks](https://mlsanity.com/paper/1509.06163/) (2015)

## Built on by (most cited first, 7 of 7)

- [Dyn-O: Building Structured World Models with Object-Centric Representations](https://mlsanity.com/paper/2507.03298/) (2025)
- [Analyzing Deep Transformer Models for Time Series Forecasting via Manifold Learning](https://mlsanity.com/paper/2410.13792/) (2024)
- [FreeSliders: Training-Free, Modality-Agnostic Concept Sliders for Fine-Grained Diffusion Control in Images, Audio, and Video](https://mlsanity.com/paper/2511.00103/) (2025)
- [Unsupervised Representation Learning from Sparse Transformation Analysis](https://mlsanity.com/paper/2410.05564/) (2024)
- [Bitrate-Controlled Diffusion for Disentangling Motion and Content in Video](https://mlsanity.com/paper/2509.08376/) (2025)
- [DiViD: Disentangled Video Diffusion for Static-Dynamic Factorization](https://mlsanity.com/paper/2507.13934/) (2025)
- [Disentanglement Beyond Static vs. Dynamic: A Benchmark and Evaluation Framework for Multi-Factor Sequential Representations](https://mlsanity.com/paper/2510.17313/) (2025)

## Abstract (arXiv)

One of the fundamental representation learning tasks is unsupervised sequential disentanglement, where latent codes of inputs are decomposed to a single static factor and a sequence of dynamic factors. To extract this latent information, existing methods condition the static and dynamic codes on the entire input sequence. Unfortunately, these models often suffer from information leakage, i.e., the dynamic vectors encode both static and dynamic information, or vice versa, leading to a non-disentangled representation. Attempts to alleviate this problem via reducing the dynamic dimension and auxiliary loss terms gain only partial success. Instead, we propose a novel and simple architecture that mitigates information leakage by offering a simple and effective subtraction inductive bias while conditioning on a single sample. Remarkably, the resulting variational framework is simpler in terms of required loss terms, hyperparameters, and data augmentation. We evaluate our method on multiple data-modality benchmarks including general time series, video, and audio, and we show beyond state-of-the-art results on generation and prediction tasks in comparison to several strong baselines.