# Long Short-Term Memory Kalman Filters:Recurrent Neural Estimators for Pose Regularization

arXiv: https://arxiv.org/abs/1708.01885 · HTML: https://mlsanity.com/paper/1708.01885/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2017.

## Built on by (most cited first, 30 of 36)

- [KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics](https://mlsanity.com/paper/2107.10043/) (2021)
- [SmoothNet: A Plug-and-Play Network for Refining Human Poses in Videos](https://mlsanity.com/paper/2112.13715/) (2021)
- [Model-Based Deep Learning: On the Intersection of Deep Learning and Optimization](https://mlsanity.com/paper/2205.02640/) (2022)
- [Learning Camera Localization via Dense Scene Matching](https://mlsanity.com/paper/2103.16792/) (2021)
- [DANSE: Data-driven Non-linear State Estimation of Model-free Process in Unsupervised Learning Setup](https://mlsanity.com/paper/2306.03897/) (2023)
- [Unsupervised Learned Kalman Filtering](https://mlsanity.com/paper/2110.09005/) (2021)
- [KFD-NeRF: Rethinking Dynamic NeRF with Kalman Filter](https://mlsanity.com/paper/2407.13185/) (2024)
- [Learning to Assimilate in Chaotic Dynamical Systems](https://mlsanity.com/paper/2111.01058/) (2021)
- [GSP-KalmanNet: Tracking Graph Signals via Neural-Aided Kalman Filtering](https://mlsanity.com/paper/2311.16602/) (2023)
- [Constrained Block Nonlinear Neural Dynamical Models](https://mlsanity.com/paper/2101.01864/) (2021)
- [Optimization or Architecture: How to Hack Kalman Filtering](https://mlsanity.com/paper/2310.00675/) (2023)
- [Interpretable Deep Feature Propagation for Early Action Recognition](https://mlsanity.com/paper/2107.05122/) (2021)
- [Neural Augmentation of Kalman Filter with Hypernetwork for Channel Tracking](https://mlsanity.com/paper/2109.12561/) (2021)
- [RFNet-4D++: Joint Object Reconstruction and Flow Estimation from 4D Point Clouds with Cross-Attention Spatio-Temporal Features](https://mlsanity.com/paper/2203.16482/) (2022)
- [Model-free Online Learning for the Kalman Filter: Forgetting Factor and Logarithmic Regret](https://mlsanity.com/paper/2505.08982/) (2025)
- [The Fragility of Noise Estimation in Kalman Filter: Optimization Can Handle Model-Misspecification](https://mlsanity.com/paper/2104.02372/) (2021)
- [Efficient Non-Compression Auto-Encoder for Driving Noise-based Road Surface Anomaly Detection](https://mlsanity.com/paper/2111.10985/) (2021)
- [AI-Aided Kalman Filters](https://mlsanity.com/paper/2410.12289/) (2024)
- [Cellular Traffic Prediction via Deep State Space Models with Attention Mechanism](https://mlsanity.com/paper/2506.15688/) (2025)
- [KARNet: Kalman Filter Augmented Recurrent Neural Network for Learning World Models in Autonomous Driving Tasks](https://mlsanity.com/paper/2305.14644/) (2023)
- [Revisiting semi-supervised training objectives for differentiable particle filters](https://mlsanity.com/paper/2405.01251/) (2024)
- [Survey of Deep Learning Methods for Inverse Problems](https://mlsanity.com/paper/2111.04731/) (2021)
- [Continuous Forecasting via Neural Eigen Decomposition](https://mlsanity.com/paper/2202.00117/) (2022)
- [Learning IMM Filter Parameters from Measurements using Gradient Descent](https://mlsanity.com/paper/2307.06618/) (2023)
- [Logarithmic Regret and Polynomial Scaling in Online Multi-step-ahead Prediction](https://mlsanity.com/paper/2511.12467/) (2025)
- [Multi-Object Tracking with Deep Learning Ensemble for Unmanned Aerial System Applications](https://mlsanity.com/paper/2110.02044/) (2021)
- [TRADE: Object Tracking with 3D Trajectory and Ground Depth Estimates for UAVs](https://mlsanity.com/paper/2210.03270/) (2022)
- [Individualized Dosing Dynamics via Neural Eigen Decomposition](https://mlsanity.com/paper/2306.14020/) (2023)
- [Data-Driven Approaches for Modelling Target Behaviour](https://mlsanity.com/paper/2410.10538/) (2024)
- [DIMM: Decoupled Multi-hierarchy Kalman Filter for 3D Object Tracking](https://mlsanity.com/paper/2505.12340/) (2025)

## Abstract (arXiv)

One-shot pose estimation for tasks such as body joint localization, camera pose estimation, and object tracking are generally noisy, and temporal filters have been extensively used for regularization. One of the most widely-used methods is the Kalman filter, which is both extremely simple and general. However, Kalman filters require a motion model and measurement model to be specified a priori, which burdens the modeler and simultaneously demands that we use explicit models that are often only crude approximations of reality. For example, in the pose-estimation tasks mentioned above, it is common to use motion models that assume constant velocity or constant acceleration, and we believe that these simplified representations are severely inhibitive. In this work, we propose to instead learn rich, dynamic representations of the motion and noise models. In particular, we propose learning these models from data using long short term memory, which allows representations that depend on all previous observations and all previous states. We evaluate our method using three of the most popular pose estimation tasks in computer vision, and in all cases we obtain state-of-the-art performance.