mlsanityData to 14 Oct 2026

Unsupervised Learning of Disentangled Representations from Video

arXiv 1705.10915 ↗First posted 31 May 2017cs.LG cs.AI cs.CVcited by 139

Not on the map. Why?

Abstract (arXiv)

We present a new model DrNET that learns disentangled image representations from video. Our approach leverages the temporal coherence of video and a novel adversarial loss to learn a representation that factorizes each frame into a stationary part and a temporally varying component. The disentangled representation can be used for a range of tasks. For example, applying a standard LSTM to the time-vary components enables prediction of future frames. We evaluate our approach on a range of synthetic and real videos, demonstrating the ability to coherently generate hundreds of steps into the future.

Key facts. Cited by 139 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 57; 2022: 28; 2023: 20; 2024: 14; 2025: 10; 2026: 9.

139 papers.

Sort:

By area:

  1. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  2. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  3. 2021Unsupervised Representation Learning for Time Series with Temporal Neighborhood Codingcited by 224
  4. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  5. 2022SimVP: Simpler yet Better Video Predictioncited by 184
  6. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 182
  7. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 175
  8. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 163
  9. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  10. 2021Image-to-Image Translation: Methods and Applicationscited by 116
  11. 2021Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regressioncited by 112
  12. 2022Disentangled Representation Learningcited by 112
38 more
  1. 2022Unsupervised Time-Series Representation Learning with Iterative Bilinear Temporal-Spectral Fusioncited by 96
  2. 2022GIRAFFE HD: A High-Resolution 3D-aware Generative Modelcited by 86
  3. 2022Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learningcited by 86
  4. 2021Deconfounded Video Moment Retrieval with Causal Interventioncited by 84
  5. 2021Decoupling Value and Policy for Generalization in Reinforcement Learningcited by 82
  6. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 76
  7. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  8. 2022When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learningcited by 65
  9. 2022Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Imagecited by 55
  10. 2022Efficient Video Prediction via Sparsely Conditioned Flow Matchingcited by 55
  11. 2021Deep Gait Recognition: A Surveycited by 54
  12. 2021Multimodal Image Synthesis and Editing: The Generative AI Eracited by 52
  13. 2021StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGANcited by 47
  14. 2022Self-Supervised Time Series Representation Learning via Cross Reconstruction Transformercited by 47
  15. 2024Usable XAI: 10 Strategies Towards Exploiting Explainability in the LLM Eracited by 46
  16. 2023Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learningcited by 45
  17. 2024Efficient Video Diffusion Models via Content-Frame Motion-Latent Decompositioncited by 42
  18. 2023LEO: Generative Latent Image Animator for Human Video Synthesiscited by 40
  19. 2021A Novel Estimator of Mutual Information for Learning to Disentangle Textual Representationscited by 37
  20. 2022Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioningcited by 37
  21. 2021EarthNet2021: A large-scale dataset and challenge for Earth surface forecasting as a guided video prediction taskcited by 34
  22. 2021Video Autoencoder: self-supervised disentanglement of static 3D structure and motioncited by 34
  23. 2021Contrastively Disentangled Sequential Variational Autoencodercited by 34
  24. 2021An Image is Worth More Than a Thousand Words: Towards Disentanglement in the Wildcited by 30
  25. 2021Position, Padding and Predictions: A Deeper Look at Position Information in CNNscited by 26
  26. 2023Multifactor Sequential Disentanglement via Structured Koopman Autoencoderscited by 24
  27. 2021Self-Supervised Disentangled Representation Learning for Third-Person Imitation Learningcited by 23
  28. 2021Disentangled Recurrent Wasserstein Autoencodercited by 22
  29. 2021Representation Disentanglement for Multi-modal brain MR Analysiscited by 22
  30. 2022SimVPv2: Towards Simple yet Powerful Spatiotemporal Predictive Learningcited by 22
  31. 2023Temporally Disentangled Representation Learning under Unknown Nonstationaritycited by 22
  32. 2021Multimodal Emotion Recognition with High-level Speech and Text Featurescited by 20
  33. 2022Learning to Refactor Action and Co-occurrence Features for Temporal Action Localizationcited by 18
  34. 2023Not Only Generative Art: Stable Diffusion for Content-Style Disentanglement in Art Analysiscited by 18
  35. 2023MMVP: Motion-Matrix-based Video Predictioncited by 18
  36. 2021Comparing Correspondences: Video Prediction with Correspondence-wise Lossescited by 16
  37. 2021Scaling-up Disentanglement for Image Translationcited by 15
  38. 2021Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translationcited by 15

50 most cited of 139 All 139 (JSON)