mlsanityData to 7 Oct 2026

S3VAE: Self-Supervised Sequential VAE for Representation Disentanglement and Data Generation

arXiv 2005.11437 ↗First posted 23 May 2020cs.CVcited by 63

Not on the map. Why?

Abstract (arXiv)

We propose a sequential variational autoencoder to learn disentangled representations of sequential data (e.g., videos and audios) under self-supervision. Specifically, we exploit the benefits of some readily accessible supervisory signals from input data itself or some off-the-shelf functional models and accordingly design auxiliary tasks for our model to utilize these signals. With the supervision of the signals, our model can easily disentangle the representation of an input sequence into static factors and dynamic factors (i.e., time-invariant and time-varying parts). Comprehensive experiments across videos and audios verify the effectiveness of our model on representation disentanglement and generation of sequential data, and demonstrate that, our model with self-supervision performs comparable to, if not better than, the fully-supervised model with ground truth labels, and outperforms state-of-the-art unsupervised models by a large margin.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 13; 2023: 12; 2024: 12; 2025: 12; 2026: 1.

63 papers.

Sort:

By area:

  1. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  2. 2021Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regressioncited by 112
  3. 2021Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoderscited by 109
  4. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  5. 2021Self-Supervised Learning Disentangled Group Representation as Featurecited by 47
  6. 2021Contrastively Disentangled Sequential Variational Autoencodercited by 34
  7. 2022Robust Disentangled Variational Speech Representation Learning for Zero-shot Voice Conversioncited by 28
  8. 2023Multifactor Sequential Disentanglement via Structured Koopman Autoencoderscited by 23
  9. 2021On Causally Disentangled Representationscited by 22
  10. 2022Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspectivecited by 21
  11. 2023Temporally Disentangled Representation Learning under Unknown Nonstationaritycited by 21
  12. 2025Robust Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoningcited by 20
38 more
  1. 2022Decoupling Local and Global Representations of Time Seriescited by 17
  2. 2021Action2video: Generating Videos of Human 3D Actionscited by 15
  3. 2024Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inferencecited by 15
  4. 2024A Survey of Generative Techniques for Spatial-Temporal Data Miningcited by 14
  5. 2021Deep Human-guided Conditional Variational Generative Modeling for Automated Urban Planningcited by 13
  6. 2023Disentangled Counterfactual Learning for Physical Audiovisual Commonsense Reasoningcited by 13
  7. 2025MoAlign: Motion-Centric Representation Alignment for Video Diffusion Modelscited by 13
  8. 2023SCD-Net: Spatiotemporal Clues Disentanglement Network for Self-supervised Skeleton-based Action Recognitioncited by 11
  9. 2025HiD-VAE: Interpretable Generative Recommendation via Hierarchical and Disentangled Semantic IDscited by 11
  10. 2022Towards Improved Zero-shot Voice Conversion with Conditional DSVAEcited by 10
  11. 2023Audio2Gestures: Generating Diverse Gestures from Audiocited by 10
  12. 2023Sample and Predict Your Latent: Modality-free Sequential Disentanglement via Contrastive Estimationcited by 10
  13. 2024Combining audio control and style transfer using latent diffusioncited by 10
  14. 2021S3RP: Self-Supervised Super-Resolution and Prediction for Advection-Diffusion Processcited by 9
  15. 2022Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAEcited by 7
  16. 2024Sequential Disentanglement by Extracting Static Information From A Single Sequence Elementcited by 7
  17. 2024Collapse-Proof Non-Contrastive Self-Supervised Learningcited by 7
  18. 2021VDSM: Unsupervised Video Disentanglement with State-Space Modeling and Deep Mixtures of Expertscited by 6
  19. 2022Towards Robust Unsupervised Disentanglement of Sequential Data -- A Case Study Using Music Audiocited by 6
  20. 2021Hamiltonian latent operators for content and motion disentanglement in image sequencescited by 5
  21. 2022Predicting Human Mobility via Self-supervised Disentanglement Learningcited by 5
  22. 2023Neural Latent Aligner: Cross-trial Alignment for Learning Representations of Complex, Naturalistic Neural Datacited by 5
  23. 2021Online Unsupervised Learning of Visual Representations and Categoriescited by 4
  24. 2022Towards Efficient Dialogue Pre-training with Transferable and Interpretable Latent Structurecited by 4
  25. 2023Learning Interpretable Low-dimensional Representation via Physical Symmetrycited by 4
  26. 2024Sequential Representation Learning via Static-Dynamic Conditional Disentanglementcited by 4
  27. 2021Video Reenactment as Inductive Bias for Content-Motion Disentanglementcited by 3
  28. 2022Disentangled and Side-aware Unsupervised Domain Adaptation for Cross-dataset Subjective Tinnitus Diagnosiscited by 3
  29. 2023Self-Supervised Learning for Data Scarcity in a Fatigue Damage Prognostic Problemcited by 3
  30. 2022Self-supervised Model Based on Masked Autoencoders Advance CT Scans Classificationcited by 2
  31. 2023Learning Concept-Based Causal Transition and Symbolic Reasoning for Visual Planningcited by 2
  32. 2023Unifying Self-Supervised Clustering and Energy-Based Modelscited by 2
  33. 2024Nonstationary Time Series Forecasting via Unknown Distribution Adaptationcited by 2
  34. 2024Unsupervised Representation Learning from Sparse Transformation Analysiscited by 2
  35. 2025L-VAE: Variational Auto-Encoder with Learnable Beta for Disentangled Representationcited by 2
  36. 2025Bitrate-Controlled Diffusion for Disentangling Motion and Content in Videocited by 2
  37. 2022Benchmarking Generative Latent Variable Models for Speechcited by 1
  38. 2022Enhancing Zero-Shot Many to Many Voice Conversion with Self-Attention VAEcited by 1

50 most cited of 63 All 63 (JSON)