# KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics

arXiv: https://arxiv.org/abs/2107.10043 · HTML: https://mlsanity.com/paper/2107.10043/

## Key facts

- Cited by 132 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 82 references that are in the map.
- First posted on arXiv 21 Jul 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Gate-Variants of Gated Recurrent Unit (GRU) Neural Networks](https://mlsanity.com/paper/1701.05923/) (2017)
- [Structured Inference Networks for Nonlinear State Space Models](https://mlsanity.com/paper/1609.09869/) (2016)
- [Deep Kalman Filters](https://mlsanity.com/paper/1511.05121/) (2015)
- [Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Data](https://mlsanity.com/paper/1605.06432/) (2016)
- [Model-Based Deep Learning](https://mlsanity.com/paper/2012.08405/) (2020)
- [A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learning](https://mlsanity.com/paper/1710.05741/) (2017)
- [Variational Sequential Monte Carlo](https://mlsanity.com/paper/1705.11140/) (2017)
- [Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces](https://mlsanity.com/paper/1905.07357/) (2019)
- [Backprop KF: Learning Discriminative Deterministic State Estimators](https://mlsanity.com/paper/1605.07148/) (2016)
- [KFNet: Learning Temporal Camera Relocalization using Kalman Filtering](https://mlsanity.com/paper/2003.10629/) (2020)
- [Long Short-Term Memory Kalman Filters:Recurrent Neural Estimators for Pose Regularization](https://mlsanity.com/paper/1708.01885/) (2017)
- [Combining Generative and Discriminative Models for Hybrid Inference](https://mlsanity.com/paper/1906.02547/) (2019)
- [Black box variational inference for state space models](https://mlsanity.com/paper/1511.07367/) (2015)
- [ViterbiNet: A Deep Learning Based Viterbi Algorithm for Symbol Detection](https://mlsanity.com/paper/1905.10750/) (2019)
- [DeepSIC: Deep Soft Interference Cancellation for Multiuser MIMO Detection](https://mlsanity.com/paper/2002.03214/) (2020)
- [State Space LSTM Models with Particle MCMC Inference](https://mlsanity.com/paper/1711.11179/) (2017)
- [Fitting a Kalman Smoother to Data](https://mlsanity.com/paper/1910.08615/) (2019)
- [Learned Factor Graphs for Inference from Stationary Time Sequences](https://mlsanity.com/paper/2006.03258/) (2020)
- [Robust Kalman Filtering under Model Perturbations](https://mlsanity.com/paper/1508.01906/) (2015)
- [On the Robustness of the Bayes and Wiener Estimators under Model Uncertainty](https://mlsanity.com/paper/1508.01904/) (2015)
- [Learning the tuned liquid damper dynamics by means of a robust EKF](https://mlsanity.com/paper/2103.03520/) (2021)

## Built on by (most cited first, 30 of 132)

- [Machine learning with data assimilation and uncertainty quantification for dynamical systems: a review](https://mlsanity.com/paper/2303.10462/) (2023)
- [Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization](https://mlsanity.com/paper/2205.02640/) (2022)
- [Deep Generative Data Assimilation in Multimodal Setting](https://mlsanity.com/paper/2404.06665/) (2024)
- [Theoretical Perspectives on Deep Learning Methods in Inverse Problems](https://mlsanity.com/paper/2206.14373/) (2022)
- [Forking Uncertainties: Reliable Prediction and Model Predictive Control with Sequence Models via Conformal Risk Control](https://mlsanity.com/paper/2310.10299/) (2023)
- [DANSE: Data-driven Non-linear State Estimation of Model-free Process in Unsupervised Learning Setup](https://mlsanity.com/paper/2306.03897/) (2023)
- [Unsupervised Learned Kalman Filtering](https://mlsanity.com/paper/2110.09005/) (2021)
- [Focus Your Attention (with Adaptive IIR Filters)](https://mlsanity.com/paper/2305.14952/) (2023)
- [KFD-NeRF: Rethinking Dynamic NeRF with Kalman Filter](https://mlsanity.com/paper/2407.13185/) (2024)
- [GSP-KalmanNet: Tracking Graph Signals via Neural-Aided Kalman Filtering](https://mlsanity.com/paper/2311.16602/) (2023)
- [Normalizing Flow-based Differentiable Particle Filters](https://mlsanity.com/paper/2403.01499/) (2024)
- [Optimization or Architecture: How to Hack Kalman Filtering](https://mlsanity.com/paper/2310.00675/) (2023)
- [Beyond Kalman Filters: Deep Learning-Based Filters for Improved Object Tracking](https://mlsanity.com/paper/2402.09865/) (2024)
- [Discussing the Spectrum of Physics-Enhanced Machine Learning; a Survey on Structural Mechanics Applications](https://mlsanity.com/paper/2310.20425/) (2023)
- [CGKN: A Deep Learning Framework for Modeling Complex Dynamical Systems and Efficient Data Assimilation](https://mlsanity.com/paper/2410.20072/) (2024)
- [Uncertainty in Data-Driven Kalman Filtering for Partially Known State-Space Models](https://mlsanity.com/paper/2110.04738/) (2021)
- [CGNSDE: Conditional Gaussian Neural Stochastic Differential Equation for Modeling Complex Systems and Data Assimilation](https://mlsanity.com/paper/2404.06749/) (2024)
- [Deep Bayesian Filter for Bayes-faithful Data Assimilation](https://mlsanity.com/paper/2405.18674/) (2024)
- [Optimal-state Dynamics Estimation for Physics-based Human Motion Capture from Videos](https://mlsanity.com/paper/2410.07795/) (2024)
- [Recursive KalmanNet: Deep Learning-Augmented Kalman Filtering for State Estimation with Consistent Uncertainty Quantification](https://mlsanity.com/paper/2506.11639/) (2025)
- [Modeling Partially Observed Nonlinear Dynamical Systems and Efficient Data Assimilation via Discrete-Time Conditional Gaussian Koopman Network](https://mlsanity.com/paper/2507.08749/) (2025)
- [Hybrid Neural Network Augmented Physics-based Models for Nonlinear Filtering](https://mlsanity.com/paper/2204.06471/) (2022)
- [Towards Flexibility and Interpretability of Gaussian Process State-Space Model](https://mlsanity.com/paper/2301.08843/) (2023)
- [Ensemble Kalman Filtering Meets Gaussian Process SSM for Non-Mean-Field and Online Inference](https://mlsanity.com/paper/2312.05910/) (2023)
- [Deep Learning-Based Robust Multi-Object Tracking via Fusion of mmWave Radar and Camera Sensors](https://mlsanity.com/paper/2407.08049/) (2024)
- [Prediction and Interpretation of Vehicle Trajectories in the Graph Spectral Domain](https://mlsanity.com/paper/2309.16702/) (2023)
- [Learning Semilinear Neural Operators : A Unified Recursive Framework For Prediction And Data Assimilation](https://mlsanity.com/paper/2402.15656/) (2024)
- [A Survey of Machine Learning Techniques for Improving Global Navigation Satellite Systems](https://mlsanity.com/paper/2406.16873/) (2024)
- [KODA: A Data-Driven Recursive Model for Time Series Forecasting and Data Assimilation using Koopman Operators](https://mlsanity.com/paper/2409.19518/) (2024)
- [Online Control-Informed Learning](https://mlsanity.com/paper/2410.03924/) (2024)

## Abstract (arXiv)

State estimation of dynamical systems in real-time is a fundamental task in signal processing. For systems that are well-represented by a fully known linear Gaussian state space (SS) model, the celebrated Kalman filter (KF) is a low complexity optimal solution. However, both linearity of the underlying SS model and accurate knowledge of it are often not encountered in practice. Here, we present KalmanNet, a real-time state estimator that learns from data to carry out Kalman filtering under non-linear dynamics with partial information. By incorporating the structural SS model with a dedicated recurrent neural network module in the flow of the KF, we retain data efficiency and interpretability of the classic algorithm while implicitly learning complex dynamics from data. We demonstrate numerically that KalmanNet overcomes non-linearities and model mismatch, outperforming classic filtering methods operating with both mismatched and accurate domain knowledge.