# Learning deep dynamical models from image pixels

arXiv: https://arxiv.org/abs/1410.7550 · HTML: https://mlsanity.com/paper/1410.7550/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2014.

## Built on by (most cited first, 7 of 7)

- [Input-to-State Stable Coupled Oscillator Networks for Closed-form Model-based Control in Latent Space](https://mlsanity.com/paper/2409.08439/) (2024)
- [Fractional Transfer Learning for Deep Model-Based Reinforcement Learning](https://mlsanity.com/paper/2108.06526/) (2021)
- [Deep Kernel Learning of Dynamical Models from High-Dimensional Noisy Data](https://mlsanity.com/paper/2208.12975/) (2022)
- [Learning Multiple Initial Solutions to Optimization Problems](https://mlsanity.com/paper/2411.02158/) (2024)
- [Cloth Manipulation Planning on Basis of Mesh Representations with Incomplete Domain Knowledge and Voxel-to-Mesh Estimation](https://mlsanity.com/paper/2103.08137/) (2021)
- [Using Gaussian Processes to Design Dynamic Experiments for Black-Box Model Discrimination under Uncertainty](https://mlsanity.com/paper/2102.03782/) (2021)
- [Learning Control-Affine Reduced-Order Models via Autoencoders](https://mlsanity.com/paper/2606.05045/) (2026)

## Abstract (arXiv)

Modeling dynamical systems is important in many disciplines, e.g., control, robotics, or neurotechnology. Commonly the state of these systems is not directly observed, but only available through noisy and potentially high-dimensional observations. In these cases, system identification, i.e., finding the measurement mapping and the transition mapping (system dynamics) in latent space can be challenging. For linear system dynamics and measurement mappings efficient solutions for system identification are available. However, in practical applications, the linearity assumptions does not hold, requiring non-linear system identification techniques. If additionally the observations are high-dimensional (e.g., images), non-linear system identification is inherently hard. To address the problem of non-linear system identification from high-dimensional observations, we combine recent advances in deep learning and system identification. In particular, we jointly learn a low-dimensional embedding of the observation by means of deep auto-encoders and a predictive transition model in this low-dimensional space. We demonstrate that our model enables learning good predictive models of dynamical systems from pixel information only.