mlsanityData to 14 Oct 2026

Contrastively Disentangled Sequential Variational Autoencoder

arXiv 2110.12091 ↗First posted 22 Oct 2021cs.LG cs.AIcited by 34

Not on the map. Why?

Abstract (arXiv)

Self-supervised disentangled representation learning is a critical task in sequence modeling. The learnt representations contribute to better model interpretability as well as the data generation, and improve the sample efficiency for downstream tasks. We propose a novel sequence representation learning method, named Contrastively Disentangled Sequential Variational Autoencoder (C-DSVAE), to extract and separate the static (time-invariant) and dynamic (time-variant) factors in the latent space. Different from previous sequential variational autoencoder methods, we use a novel evidence lower bound which maximizes the mutual information between the input and the latent factors, while penalizes the mutual information between the static and dynamic factors. We leverage contrastive estimations of the mutual information terms in training, together with simple yet effective augmentation techniques, to introduce additional inductive biases. Our experiments show that C-DSVAE significantly outperforms the previous state-of-the-art methods on multiple metrics.

Key facts. Cited by 34 papers in the map (data through 14 Oct 2026). Builds on 43 of its 59 references that are in the map. First posted on arXiv 22 Oct 2021.

Builds on

43 of 59 references.

References · 43

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2013Auto-Encoding Variational Bayescited by 4,053
  7. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  8. 2020Supervised Contrastive Learningcited by 2,829
  9. 2018Deep contextualized word representationscited by 2,290
  10. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  11. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 941
  12. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
31 more
  1. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  2. 2018Disentangling by Factorisingcited by 616
  3. 2015Generating Sentences from a Continuous Spacecited by 612
  4. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 524
  5. 2019Variational Autoencoders and Nonlinear ICA: A Unifying Frameworkcited by 503
  6. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  7. 2016Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICAcited by 273
  8. 2017VAE with a VampPriorcited by 248
  9. 2018Stochastic Video Generation with a Learned Priorcited by 245
  10. 2020Weakly-Supervised Disentanglement Without Compromisescited by 239
  11. 2017Decomposing Motion and Content for Natural Video Sequence Predictioncited by 169
  12. 2018Isolating Sources of Disentanglement in Variational Autoencoderscited by 167
  13. 2015Deep Kalman Filterscited by 166
  14. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 163
  15. 2017Unsupervised Learning of Disentangled Representations from Videocited by 139
  16. 2018Recent Advances in Autoencoder-Based Representation Learningcited by 133
  17. 2018Disentangled Sequential Autoencodercited by 120
  18. 2017A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learningcited by 107
  19. 2017A Generative Adversarial Approach for Zero-Shot Learning from Noisy Textscited by 96
  20. 2017Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Datacited by 93
  21. 2019The Incomplete Rosetta Stone Problem: Identifiability Results for Multi-View Nonlinear ICAcited by 86
  22. 2019Generative Pre-Training for Speech with Autoregressive Predictive Codingcited by 82
  23. 2020S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generationcited by 63
  24. 2020Improving Disentangled Text Representation Learning with Information-Theoretic Guidancecited by 52
  25. 2017Z-Forcing: Training Stochastic Recurrent Networkscited by 51
  26. 2020Unsupervised Pre-training of Bidirectional Speech Encoders via Masked Reconstructioncited by 38
  27. 2018Variational Autoencoder with Implicit Optimal Priorscited by 33
  28. 2020Feedback Recurrent Autoencoder for Video Compressioncited by 30
  29. 2021Disentangled Recurrent Wasserstein Autoencodercited by 22
  30. 2020Representation Learning for Sequence Data with Deep Autoencoding Predictive Componentscited by 9
  31. 2017Inception Score, Label Smoothing, Gradient Vanishing and -log(D(x)) Alternativecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 13; 2023: 5; 2024: 7; 2025: 8; 2026: 1.

34 papers.

Sort:

By area:

  1. 2023Multifactor Sequential Disentanglement via Structured Koopman Autoencoderscited by 24
  2. 2023Temporally Disentangled Representation Learning under Unknown Nonstationaritycited by 22
  3. 2022Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspectivecited by 21
  4. 2025Robust Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoningcited by 20
  5. 2022ContrastVAE: Contrastive Variational AutoEncoder for Sequential Recommendationcited by 19
  6. 2022Decoupling Local and Global Representations of Time Seriescited by 17
  7. 2022Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamicscited by 14
  8. 2023Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoningcited by 13
  9. 2025Dyn-O: Building Structured World Models with Object-Centric Representationscited by 13
  10. 2022Towards Improved Zero-shot Voice Conversion with Conditional DSVAEcited by 10
  11. 2023Sample and Predict Your Latent: Modality-free Sequential Disentanglement via Contrastive Estimationcited by 10
  12. 2024Combining audio control and style transfer using latent diffusioncited by 10
22 more
  1. 2022Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAEcited by 7
  2. 2022Relative Behavioral Attributes: Filling the Gap between Symbolic Goal Specification and Reward Learning from Human Preferencescited by 7
  3. 2024Sequential Disentanglement by Extracting Static Information From A Single Sequence Elementcited by 7
  4. 2022Towards Robust Unsupervised Disentanglement of Sequential Data -- A Case Study Using Music Audiocited by 6
  5. 2024Enhancing Evolving Domain Generalization through Dynamic Latent Representationscited by 6
  6. 2022Leveraging Relational Information for Learning Weakly Disentangled Representationscited by 5
  7. 2022Predicting Human Mobility via Self-supervised Disentanglement Learningcited by 5
  8. 2022Retrieval-based Disentangled Representation Learning with Natural Language Supervisioncited by 5
  9. 2022DisPositioNet: Disentangled Pose and Identity in Semantic Image Manipulationcited by 4
  10. 2023Learning Interpretable Low-dimensional Representation via Physical Symmetrycited by 4
  11. 2024Sequential Representation Learning via Static-Dynamic Conditional Disentanglementcited by 4
  12. 2025Disentanglement of Variations with Multimodal Generative Modelingcited by 4
  13. 2025FreeSliders: Training-Free, Modality-Agnostic Concept Sliders for Fine-Grained Diffusion Control in Images, Audio, and Videocited by 4
  14. 2024Unsupervised Disentanglement of Content and Style via Variance-Invariance Constraintscited by 2
  15. 2024Unsupervised Representation Learning from Sparse Transformation Analysiscited by 2
  16. 2025Bitrate-Controlled Diffusion for Disentangling Motion and Content in Videocited by 2
  17. 2025DiViD: Disentangled Video Diffusion for Static-Dynamic Factorizationcited by 1
  18. 2025Disentanglement Beyond Static vs. Dynamic: A Benchmark and Evaluation Framework for Multi-Factor Sequential Representationscited by 1
  19. 2022Modular Representations for Weak Disentanglementcited by 0
  20. 2024Unsupervised Representation Learning of Complex Time Series for Maneuverability State Identification in Smart Mobilitycited by 0
  21. 2025Multiple Invertible and Partial-Equivariant Function for Latent Vector Transformation to Enhance Disentanglement in VAEscited by 0
  22. 2026Koopman Regularized Deep Speech Disentanglement for Speaker Verificationcited by 0