# Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spaces

arXiv: https://arxiv.org/abs/1905.07357 · HTML: https://mlsanity.com/paper/1905.07357/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2019.

## Built on by (most cited first, 30 of 67)

- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics](https://mlsanity.com/paper/2107.10043/) (2021)
- [Modeling Irregular Time Series with Continuous Recurrent Units](https://mlsanity.com/paper/2111.11344/) (2021)
- [Generalised Latent Assimilation in Heterogeneous Reduced Spaces with Machine Learning Surrogate Models](https://mlsanity.com/paper/2204.03497/) (2022)
- [DANSE: Data-driven Non-linear State Estimation of Model-free Process in Unsupervised Learning Setup](https://mlsanity.com/paper/2306.03897/) (2023)
- [Unsupervised Learned Kalman Filtering](https://mlsanity.com/paper/2110.09005/) (2021)
- [S7: Selective and Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2410.03464/) (2024)
- [Latent SDEs on Homogeneous Spaces](https://mlsanity.com/paper/2306.16248/) (2023)
- [SSDNet: State Space Decomposition Neural Network for Time Series Forecasting](https://mlsanity.com/paper/2112.10251/) (2021)
- [Grounding Graph Network Simulators using Physical Sensor Observations](https://mlsanity.com/paper/2302.11864/) (2023)
- [On Uncertainty in Deep State Space Models for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2210.09256/) (2022)
- [Disentangling Voice and Content with Self-Supervision for Speaker Recognition](https://mlsanity.com/paper/2310.01128/) (2023)
- [KFD-NeRF: Rethinking Dynamic NeRF with Kalman Filter](https://mlsanity.com/paper/2407.13185/) (2024)
- [Learning to Assimilate in Chaotic Dynamical Systems](https://mlsanity.com/paper/2111.01058/) (2021)
- [Efficient Dynamics Modeling in Interactive Environments with Koopman Theory](https://mlsanity.com/paper/2306.11941/) (2023)
- [GSP-KalmanNet: Tracking Graph Signals via Neural-Aided Kalman Filtering](https://mlsanity.com/paper/2311.16602/) (2023)
- [Amortized Control of Continuous State Space Feynman-Kac Model for Irregular Time Series](https://mlsanity.com/paper/2410.05602/) (2024)
- [KalMamba: Towards Efficient Probabilistic State Space Models for RL under Uncertainty](https://mlsanity.com/paper/2406.15131/) (2024)
- [Hidden Parameter Recurrent State Space Models For Changing Dynamics Scenarios](https://mlsanity.com/paper/2206.14697/) (2022)
- [Multi Time Scale World Models](https://mlsanity.com/paper/2310.18534/) (2023)
- [Kalman Linear Attention: Parallel Bayesian Filtering For Efficient Language Modelling and State Tracking](https://mlsanity.com/paper/2602.10743/) (2026)
- [PILOT: Introducing Transformers for Probabilistic Sound Event Localization](https://mlsanity.com/paper/2106.03903/) (2021)
- [Temporal Test-Time Adaptation with State-Space Models](https://mlsanity.com/paper/2407.12492/) (2024)
- [Learning Dynamical Systems from Noisy Sensor Measurements using Multiple Shooting](https://mlsanity.com/paper/2106.11712/) (2021)
- [Uncertainty in Data-Driven Kalman Filtering for Partially Known State-Space Models](https://mlsanity.com/paper/2110.04738/) (2021)
- [MILD: Multimodal Interactive Latent Dynamics for Learning Human-Robot Interaction](https://mlsanity.com/paper/2210.12418/) (2022)
- [Recurrent Neural Networks and Universal Approximation of Bayesian Filters](https://mlsanity.com/paper/2211.00335/) (2022)
- [Uncertainty Representations in State-Space Layers for Deep Reinforcement Learning under Partial Observability](https://mlsanity.com/paper/2409.16824/) (2024)
- [On the Stochastic Stability of Deep Markov Models](https://mlsanity.com/paper/2111.04601/) (2021)
- [Black-Box vs. Gray-Box: A Case Study on Learning Table Tennis Ball Trajectory Prediction with Spin and Impacts](https://mlsanity.com/paper/2305.15189/) (2023)

## Abstract (arXiv)

In order to integrate uncertainty estimates into deep time-series modelling, Kalman Filters (KFs) (Kalman et al., 1960) have been integrated with deep learning models, however, such approaches typically rely on approximate inference techniques such as variational inference which makes learning more complex and often less scalable due to approximation errors. We propose a new deep approach to Kalman filtering which can be learned directly in an end-to-end manner using backpropagation without additional approximations. Our approach uses a high-dimensional factorized latent state representation for which the Kalman updates simplify to scalar operations and thus avoids hard to backpropagate, computationally heavy and potentially unstable matrix inversions. Moreover, we use locally linear dynamic models to efficiently propagate the latent state to the next time step. The resulting network architecture, which we call Recurrent Kalman Network (RKN), can be used for any time-series data, similar to a LSTM (Hochreiter & Schmidhuber, 1997) but uses an explicit representation of uncertainty. As shown by our experiments, the RKN obtains much more accurate uncertainty estimates than an LSTM or Gated Recurrent Units (GRUs) (Cho et al., 2014) while also showing a slightly improved prediction performance and outperforms various recent generative models on an image imputation task.