mlsanityData to 14 Oct 2026

A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning

arXiv 1710.05741 ↗First posted 16 Oct 2017stat.ML cs.LGcited by 107

Not on the map. Why?

Abstract (arXiv)

This paper takes a step towards temporal reasoning in a dynamically changing video, not in the pixel space that constitutes its frames, but in a latent space that describes the non-linear dynamics of the objects in its world. We introduce the Kalman variational auto-encoder, a framework for unsupervised learning of sequential data that disentangles two latent representations: an object's representation, coming from a recognition model, and a latent state describing its dynamics. As a result, the evolution of the world can be imagined and missing data imputed, both without the need to generate high dimensional frames at each time step. The model is trained end-to-end on videos of a variety of simulated physical systems, and outperforms competing methods in generative and missing data imputation tasks.

Key facts. Cited by 107 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 31; 2022: 20; 2023: 13; 2024: 17; 2025: 13; 2026: 13.

107 papers.

Sort:

By area:

  1. 2021Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecastingcited by 337
  2. 2022Diffusion-based Time Series Imputation and Forecasting with Structured State Space Modelscited by 198
  3. 2021KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamicscited by 132
  4. 2021Modeling Irregular Time Series with Continuous Recurrent Unitscited by 103
  5. 2023Generative Time Series Forecasting with Diffusion, Denoise, and Disentanglementcited by 90
  6. 2022Temporally Disentangled Representation Learningcited by 61
  7. 2022Criteria for Classifying Forecasting Methodscited by 48
  8. 2021ScoreGrad: Multivariate Probabilistic Time Series Forecasting with Continuous Energy-based Generative Modelscited by 43
  9. 2023Deep networks for system identification: a Surveycited by 37
  10. 2021Contrastively Disentangled Sequential Variational Autoencodercited by 34
  11. 2021Time Series Anomaly Detection for Cyber-Physical Systems via Neural System Identification and Bayesian Filteringcited by 31
  12. 2023Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Seriescited by 29
38 more
  1. 2021Unsupervised Speech Enhancement using Dynamical Variational Auto-Encoderscited by 28
  2. 2021Auto-differentiable Ensemble Kalman Filterscited by 28
  3. 2023Multifactor Sequential Disentanglement via Structured Koopman Autoencoderscited by 24
  4. 2023DANSE: Data-driven Non-linear State Estimation of Model-free Process in Unsupervised Learning Setupcited by 19
  5. 2022Unsupervised Representation Learning in Deep Reinforcement Learning: A Reviewcited by 18
  6. 2021Unsupervised Learned Kalman Filteringcited by 17
  7. 2021Deep Explicit Duration Switching Models for Time Seriescited by 17
  8. 2021RNN with Particle Flow for Probabilistic Spatio-temporal Forecastingcited by 16
  9. 2022Markovian Gaussian Process Variational Autoencoderscited by 16
  10. 2021Mind the Gap when Conditioning Amortised Inference in Sequential Latent-Variable Modelscited by 14
  11. 2022Conditional Generation of Medical Time Series for Extrapolation to Underrepresented Populationscited by 14
  12. 2022On Uncertainty in Deep State Space Models for Model-Based Reinforcement Learningcited by 13
  13. 2021Physics-guided Deep Markov Models for Learning Nonlinear Dynamical Systems with Uncertaintycited by 11
  14. 2024KFD-NeRF: Rethinking Dynamic NeRF with Kalman Filtercited by 11
  15. 2023Efficient Dynamics Modeling in Interactive Environments with Koopman Theorycited by 10
  16. 2024Amortized Control of Continuous State Space Feynman-Kac Model for Irregular Time Seriescited by 10
  17. 2023On the Identifiability of Switching Dynamical Systemscited by 9
  18. 2024KalMamba: Towards Efficient Probabilistic State Space Models for RL under Uncertaintycited by 9
  19. 2021Disentangled Generative Models for Robust Prediction of System Dynamicscited by 8
  20. 2022Hidden Parameter Recurrent State Space Models For Changing Dynamics Scenarioscited by 8
  21. 2022Neural Extended Kalman Filters for Learning and Predicting Dynamics of Structural Systemscited by 8
  22. 2024Temporal Test-Time Adaptation with State-Space Modelscited by 7
  23. 2025Continuous Locomotive Crowd Behavior Generationcited by 7
  24. 2021Data-driven Analysis for Understanding Team Sports Behaviorscited by 6
  25. 2021Explore the Context: Optimal Data Collection for Context-Conditional Dynamics Modelscited by 6
  26. 2021VDSM: Unsupervised Video Disentanglement with State-Space Modeling and Deep Mixtures of Expertscited by 6
  27. 2021Learning Dynamical Systems from Noisy Sensor Measurements using Multiple Shootingcited by 6
  28. 2022Deep Kernel Learning of Dynamical Models from High-Dimensional Noisy Datacited by 6
  29. 2024Deep Bayesian Filter for Bayes-faithful Data Assimilationcited by 6
  30. 2021Hamiltonian latent operators for content and motion disentanglement in image sequencescited by 5
  31. 2022Phi-DVAE: Physics-Informed Dynamical Variational Autoencoders for Unstructured Data Assimilationcited by 5
  32. 2024Uncertainty Representations in State-Space Layers for Deep Reinforcement Learning under Partial Observabilitycited by 5
  33. 2022Learning Sequential Latent Variable Models from Multimodal Time Series Datacited by 4
  34. 2023Entity Aware Modelling: A Surveycited by 4
  35. 2023Black-Box vs. Gray-Box: A Case Study on Learning Table Tennis Ball Trajectory Prediction with Spin and Impactscited by 4
  36. 2024Balanced Neural ODEs: nonlinear model order reduction and Koopman operator approximationscited by 4
  37. 2021Segmenting Hybrid Trajectories using Latent ODEscited by 3
  38. 2021Self-Supervised Inference in State-Space Modelscited by 3

50 most cited of 107 All 107 (JSON)