# DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction

arXiv: https://arxiv.org/abs/1908.03918 · HTML: https://mlsanity.com/paper/1908.03918/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2019.

## Built on by (most cited first, 13 of 13)

- [Deep Learning for Inertial Positioning: A Survey](https://mlsanity.com/paper/2303.03757/) (2023)
- [Deep Learning for Visual Localization and Mapping: A Survey](https://mlsanity.com/paper/2308.14039/) (2023)
- [VDSM: Unsupervised Video Disentanglement with State-Space Modeling and Deep Mixtures of Experts](https://mlsanity.com/paper/2103.07292/) (2021)
- [Conditional Goal-oriented Trajectory Prediction for Interacting Vehicles with Vectorized Representation](https://mlsanity.com/paper/2210.15449/) (2022)
- [Cellular Traffic Prediction via Deep State Space Models with Attention Mechanism](https://mlsanity.com/paper/2506.15688/) (2025)
- [Recovering a Molecule's 3D Dynamics from Liquid-phase Electron Microscopy Movies](https://mlsanity.com/paper/2308.11927/) (2023)
- [FishMOT: A Simple and Effective Method for Fish Tracking Based on IoU Matching](https://mlsanity.com/paper/2309.02975/) (2023)
- [CaTs and DAGs: Integrating Directed Acyclic Graphs with Transformers for Causally Constrained Predictions](https://mlsanity.com/paper/2410.14485/) (2024)
- [Lost in Tracking Translation: A Comprehensive Analysis of Visual SLAM in Human-Centered XR and IoT Ecosystems](https://mlsanity.com/paper/2411.07146/) (2024)
- [Design and Experimental Test of Datatic Approximate Optimal Filter in Nonlinear Dynamic Systems](https://mlsanity.com/paper/2505.07043/) (2025)
- [Motion Estimation for Multi-Object Tracking using KalmanNet with Semantic-Independent Encoding](https://mlsanity.com/paper/2509.11323/) (2025)
- [Coordinate ascent neural Kalman-MLE for state estimation](https://mlsanity.com/paper/2511.01855/) (2025)
- [Model-Free Neural Filtering: A Comparison with Classical Filters in Nonlinear Systems](https://mlsanity.com/paper/2601.21266/) (2026)

## Abstract (arXiv)

Dynamical models estimate and predict the temporal evolution of physical systems. State Space Models (SSMs) in particular represent the system dynamics with many desirable properties, such as being able to model uncertainty in both the model and measurements, and optimal (in the Bayesian sense) recursive formulations e.g. the Kalman Filter. However, they require significant domain knowledge to derive the parametric form and considerable hand-tuning to correctly set all the parameters. Data driven techniques e.g. Recurrent Neural Networks have emerged as compelling alternatives to SSMs with wide success across a number of challenging tasks, in part due to their ability to extract relevant features from rich inputs. They however lack interpretability and robustness to unseen conditions. In this work, we present DynaNet, a hybrid deep learning and time-varying state-space model which can be trained end-to-end. Our neural Kalman dynamical model allows us to exploit the relative merits of each approach. We demonstrate state-of-the-art estimation and prediction on a number of physically challenging tasks, including visual odometry, sensor fusion for visual-inertial navigation and pendulum control. In addition we show how DynaNet can indicate failures through investigation of properties such as the rate of innovation (Kalman Gain).