# Learning Hybrid Dynamics Models With Simulator-Informed Latent States

arXiv: https://arxiv.org/abs/2309.02873 · HTML: https://mlsanity.com/paper/2309.02873/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 43 references that are in the map.
- First posted on arXiv 6 Sep 2023.

## Builds on (most cited first, 24 of 24)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Universal Differential Equations for Scientific Machine Learning](https://mlsanity.com/paper/2001.04385/) (2020)
- [Symplectic Recurrent Neural Networks](https://mlsanity.com/paper/1909.13334/) (2019)
- [Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting](https://mlsanity.com/paper/2010.04456/) (2020)
- [Stable Recurrent Models](https://mlsanity.com/paper/1805.10369/) (2018)
- [Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Models](https://mlsanity.com/paper/1501.03291/) (2015)
- [Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profiles](https://mlsanity.com/paper/2001.11086/) (2020)
- [Physics-Integrated Variational Autoencoders for Robust and Interpretable Generative Modeling](https://mlsanity.com/paper/2102.13156/) (2021)
- [Integrating Expert ODEs into Neural ODEs: Pharmacology and Disease Progression](https://mlsanity.com/paper/2106.02875/) (2021)
- [Learning To Simulate](https://mlsanity.com/paper/1810.02513/) (2018)
- [Generative ODE Modeling with Known Unknowns](https://mlsanity.com/paper/2003.10775/) (2020)
- [On the stability properties of Gated Recurrent Units neural networks](https://mlsanity.com/paper/2011.06806/) (2020)
- [Learning Stable Deep Dynamics Models for Partially Observed or Delayed Dynamical Systems](https://mlsanity.com/paper/2110.14296/) (2021)
- [Robust Hybrid Learning With Expert Augmentation](https://mlsanity.com/paper/2202.03881/) (2022)
- [Towards gain tuning for numerical KKL observers](https://mlsanity.com/paper/2204.00318/) (2022)
- [Learning Constrained Dynamics with Gauss Principle adhering Gaussian Processes](https://mlsanity.com/paper/2004.11238/) (2020)
- [Learning-based Design of Luenberger Observers for Autonomous Nonlinear Systems](https://mlsanity.com/paper/2210.01476/) (2022)
- [The Statistical Recurrent Unit](https://mlsanity.com/paper/1703.00381/) (2017)
- [Deep KKL: Data-driven Output Prediction for Non-Linear Systems](https://mlsanity.com/paper/2103.12443/) (2021)
- [Recognition Models to Learn Dynamics from Partial Observations with Neural ODEs](https://mlsanity.com/paper/2205.12550/) (2022)
- [Luenberger observers for discrete-time nonlinear systems](https://mlsanity.com/paper/2002.02674/) (2020)
- [Combining Slow and Fast: Complementary Filtering for Dynamics Learning](https://mlsanity.com/paper/2302.13754/) (2023)

## Built on by (most cited first, 1 of 1)

- [A Comprehensive Review of Generative Physical Artificial Intelligence](https://mlsanity.com/paper/2609.18111/) (2026)

## Abstract (arXiv)

Dynamics model learning deals with the task of inferring unknown dynamics from measurement data and predicting the future behavior of the system. A typical approach to address this problem is to train recurrent models. However, predictions with these models are often not physically meaningful. Further, they suffer from deteriorated behavior over time due to accumulating errors. Often, simulators building on first principles are available being physically meaningful by design. However, modeling simplifications typically cause inaccuracies in these models. Consequently, hybrid modeling is an emerging trend that aims to combine the best of both worlds. In this paper, we propose a new approach to hybrid modeling, where we inform the latent states of a learned model via a black-box simulator. This allows to control the predictions via the simulator preventing them from accumulating errors. This is especially challenging since, in contrast to previous approaches, access to the simulator's latent states is not available. We tackle the task by leveraging observers, a well-known concept from control theory, inferring unknown latent states from observations and dynamics over time. In our learning-based setting, we jointly learn the dynamics and an observer that infers the latent states via the simulator. Thus, the simulator constantly corrects the latent states, compensating for modeling mismatch caused by learning. To maintain flexibility, we train an RNN-based residuum for the latent states that cannot be informed by the simulator.