mlsanityData to 7 Oct 2026

Disentangled Sequential Autoencoder

arXiv 1803.02991 ↗First posted 8 Mar 2018cs.LGcited by 117

Not on the map. Why?

Abstract (arXiv)

We present a VAE architecture for encoding and generating high dimensional sequential data, such as video or audio. Our deep generative model learns a latent representation of the data which is split into a static and dynamic part, allowing us to approximately disentangle latent time-dependent features (dynamics) from features which are preserved over time (content). This architecture gives us partial control over generating content and dynamics by conditioning on either one of these sets of features. In our experiments on artificially generated cartoon video clips and voice recordings, we show that we can convert the content of a given sequence into another one by such content swapping. For audio, this allows us to convert a male speaker into a female speaker and vice versa, while for video we can separately manipulate shapes and dynamics. Furthermore, we give empirical evidence for the hypothesis that stochastic RNNs as latent state models are more efficient at compressing and generating long sequences than deterministic ones, which may be relevant for applications in video compression.

Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 23; 2022: 32; 2023: 26; 2024: 19; 2025: 12; 2026: 5.

117 papers.

Sort:

By area:

  1. 2023LAVIE: High-Quality Video Generation with Cascaded Latent Diffusion Modelscited by 349
  2. 2022Diffusion Probabilistic Modeling for Video Generationcited by 237
  3. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 172
  4. 2022Disentangled Representation Learningcited by 111
  5. 2024EgoExoLearn: A Dataset for Bridging Asynchronous Ego- and Exo-centric View of Procedural Activities in Real Worldcited by 93
  6. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  7. 2021Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modelingcited by 59
  8. 2022Temporally Disentangled Representation Learningcited by 59
  9. 2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learningcited by 45
  10. 2023LEO: Generative Latent Image Animator for Human Video Synthesiscited by 40
  11. 2021Contrastively Disentangled Sequential Variational Autoencodercited by 34
  12. 2023Uncertainty-aware State Space Transformer for Egocentric 3D Hand Trajectory Forecastingcited by 33
38 more
  1. 2021Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoderscited by 28
  2. 2022Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversioncited by 28
  3. 2023Multifactor Sequential Disentanglement via Structured Koopman Autoencoderscited by 23
  4. 2021Disentangled Recurrent Wasserstein Autoencodercited by 22
  5. 2022Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspectivecited by 21
  6. 2023Temporally Disentangled Representation Learning under Unknown Nonstationaritycited by 21
  7. 2022Controllable Data Generation by Deep Learning: A Reviewcited by 20
  8. 2024CaRiNG: Learning Temporal Causal Representation under Non-Invertible Generation Processcited by 20
  9. 2021Online Variational Filtering and Parameter Learningcited by 19
  10. 2022Generalizing to Evolving Domains with Latent Structure-Aware Sequential Autoencodercited by 18
  11. 2022Learning to Refactor Action and Co-occurrence Features for Temporal Action Localizationcited by 18
  12. 2022Decoupling Local and Global Representations of Time Seriescited by 17
  13. 2024MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videoscited by 17
  14. 2022HiT-DVAE: Human Motion Generation via Hierarchical Transformer Dynamical VAEcited by 16
  15. 2022Markovian Gaussian Process Variational Autoencoderscited by 16
  16. 2022Conditional Generation of Medical Time Series for Extrapolation to Underrepresented Populationscited by 14
  17. 2024Learning Interpretable Hierarchical Dynamical Systems Models from Time Series Datacited by 14
  18. 2022Learning Latent Causal Dynamicscited by 13
  19. 2022On Uncertainty in Deep State Space Models for Model-Based Reinforcement Learningcited by 13
  20. 2023Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoningcited by 13
  21. 2024Deciphering the Role of Representation Disentanglement: Investigating Compositional Generalization in CLIP Modelscited by 13
  22. 2022Gacs-Korner Common Information Variational Autoencodercited by 12
  23. 2024Causal Temporal Representation Learning with Nonstationary Sparse Transitioncited by 12
  24. 2025Dyn-O: Building Structured World Models with Object-Centric Representationscited by 12
  25. 2023Revisiting Structured Variational Autoencoderscited by 11
  26. 2022Towards Improved Zero-shot Voice Conversion with Conditional DSVAEcited by 10
  27. 2023Sample and Predict Your Latent: Modality-free Sequential Disentanglement via Contrastive Estimationcited by 10
  28. 2021Insights from Generative Modeling for Neural Video Compressioncited by 9
  29. 2023On the Identifiability of Switching Dynamical Systemscited by 9
  30. 2024KalMamba: Towards Efficient Probabilistic State Space Models for RL under Uncertaintycited by 9
  31. 2021Unsupervised Learning of Neurosymbolic Encoderscited by 8
  32. 2021Disentangled Generative Models for Robust Prediction of System Dynamicscited by 8
  33. 2021Many-to-Many Voice Conversion based Feature Disentanglement using Variational Autoencodercited by 7
  34. 2022Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAEcited by 7
  35. 2024Sequential Disentanglement by Extracting Static Information From A Single Sequence Elementcited by 7
  36. 2021VDSM: Unsupervised Video Disentanglement with State-Space Modeling and Deep Mixtures of Expertscited by 6
  37. 2022Controllable Video Generation through Global and Local Motion Dynamicscited by 6
  38. 2022Towards Robust Unsupervised Disentanglement of Sequential Data -- A Case Study Using Music Audiocited by 6

50 most cited of 117 All 117 (JSON)