# Maximum Likelihood Learning of Latent Dynamics Without Reconstruction

arXiv: https://arxiv.org/abs/2505.23569 · HTML: https://mlsanity.com/paper/2505.23569/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 29 May 2025.

## Builds on (most cited first, 20 of 20)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review](https://mlsanity.com/paper/1805.00909/) (2018)
- [Learning Invariant Representations for Reinforcement Learning without Reconstruction](https://mlsanity.com/paper/2006.10742/) (2020)
- [DRAW: A Recurrent Neural Network For Image Generation](https://mlsanity.com/paper/1502.04623/) (2015)
- [Deep Kalman Filters](https://mlsanity.com/paper/1511.05121/) (2015)
- [Inference Suboptimality in Variational Autoencoders](https://mlsanity.com/paper/1801.03558/) (2018)
- [Composing graphical models with neural networks for structured representations and fast inference](https://mlsanity.com/paper/1603.06277/) (2016)
- [A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning](https://mlsanity.com/paper/1710.05741/) (2017)
- [Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces](https://mlsanity.com/paper/1905.07357/) (2019)
- [Optimizing Neural Networks via Koopman Operator Theory](https://mlsanity.com/paper/2006.02361/) (2020)
- [Probabilistic Contrastive Learning Recovers the Correct Aleatoric Uncertainty of Ambiguous Inputs](https://mlsanity.com/paper/2302.02865/) (2023)
- [Collapsed Amortized Variational Inference for Switching Nonlinear Dynamical Systems](https://mlsanity.com/paper/1910.09588/) (2019)
- [Stochastic Optimal Control as Approximate Input Inference](https://mlsanity.com/paper/1910.03003/) (2019)
- [Integrating Contrastive Learning with Dynamic Models for Reinforcement Learning from Images](https://mlsanity.com/paper/2203.01810/) (2022)
- [Revisiting Structured Variational Autoencoders](https://mlsanity.com/paper/2305.16543/) (2023)
- [Self-supervised contrastive learning performs non-linear system identification](https://mlsanity.com/paper/2410.14673/) (2024)
- [LatentSLAM: unsupervised multi-sensor representation learning for localization and mapping](https://mlsanity.com/paper/2105.03265/) (2021)
- [Unsupervised representation learning with recognition-parametrised probabilistic models](https://mlsanity.com/paper/2209.05661/) (2022)

## Built on by (most cited first, 3 of 3)

- [Understanding Self-Supervised Learning via Latent Distribution Matching](https://mlsanity.com/paper/2605.03517/) (2026)
- [RAMP: Recognition parametrisation by Amortised Message Passing](https://mlsanity.com/paper/2607.18883/) (2026)
- [Learning Macroscopic Dynamics without Reconstructing Microscopic States](https://mlsanity.com/paper/2609.37392/) (2026)

## Abstract (arXiv)

We introduce a novel unsupervised learning method for time series data with latent dynamical structure: the recognition-parametrized Gaussian state space model (RP-GSSM). The RP-GSSM is a probabilistic model that learns Markovian Gaussian latents explaining statistical dependence between observations at different time steps, combining the intuition of contrastive methods with the flexible tools of probabilistic generative models. Unlike contrastive approaches, the RP-GSSM is a valid probabilistic model learned via maximum likelihood. Unlike generative approaches, the RP-GSSM has no need for an explicit network mapping from latents to observations, allowing it to focus model capacity on inference of latents. The model is both tractable and expressive: it admits exact inference thanks to its jointly Gaussian latent prior, while maintaining expressivity with an arbitrarily nonlinear neural network link between observations and latents. These qualities allow the RP-GSSM to learn task-relevant latents without ad-hoc regularization, auxiliary losses, or optimizer scheduling. We show how this approach outperforms alternatives on problems that include learning nonlinear stochastic dynamics from video, with or without background distractors. Our results position the RP-GSSM as a useful foundation model for a variety of downstream applications.