# Backprop KF: Learning Discriminative Deterministic State Estimators

arXiv: https://arxiv.org/abs/1605.07148 · HTML: https://mlsanity.com/paper/1605.07148/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 May 2016.

## Built on by (most cited first, 30 of 47)

- [KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamics](https://mlsanity.com/paper/2107.10043/) (2021)
- [Differentiable SLAM-net: Learning Particle SLAM for Visual Navigation](https://mlsanity.com/paper/2105.07593/) (2021)
- [Unsupervised Learned Kalman Filtering](https://mlsanity.com/paper/2110.09005/) (2021)
- [Differentiable Particle Filters through Conditional Normalizing Flow](https://mlsanity.com/paper/2107.00488/) (2021)
- [Probabilistic 3D Multi-Object Cooperative Tracking for Autonomous Driving via Differentiable Multi-Sensor Kalman Filter](https://mlsanity.com/paper/2309.14655/) (2023)
- [Deep Learning for Visual Localization and Mapping: A Survey](https://mlsanity.com/paper/2308.14039/) (2023)
- [On Uncertainty in Deep State Space Models for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2210.09256/) (2022)
- [CTIN: Robust Contextual Transformer Network for Inertial Navigation](https://mlsanity.com/paper/2112.02143/) (2021)
- [Adaptive Kalman-Informed Transformer](https://mlsanity.com/paper/2401.09987/) (2024)
- [KFD-NeRF: Rethinking Dynamic NeRF with Kalman Filter](https://mlsanity.com/paper/2407.13185/) (2024)
- [Learning to Assimilate in Chaotic Dynamical Systems](https://mlsanity.com/paper/2111.01058/) (2021)
- [Efficient Dynamics Modeling in Interactive Environments with Koopman Theory](https://mlsanity.com/paper/2306.11941/) (2023)
- [Normalizing Flow-based Differentiable Particle Filters](https://mlsanity.com/paper/2403.01499/) (2024)
- [KalMamba: Towards Efficient Probabilistic State Space Models for RL under Uncertainty](https://mlsanity.com/paper/2406.15131/) (2024)
- [Hidden Parameter Recurrent State Space Models For Changing Dynamics Scenarios](https://mlsanity.com/paper/2206.14697/) (2022)
- [Neural Extended Kalman Filters for Learning and Predicting Dynamics of Structural Systems](https://mlsanity.com/paper/2210.04165/) (2022)
- [PILOT: Introducing Transformers for Probabilistic Sound Event Localization](https://mlsanity.com/paper/2106.03903/) (2021)
- [Accurate deep learning-based filtering for chaotic dynamics by identifying instabilities without an ensemble](https://mlsanity.com/paper/2408.04739/) (2024)
- [Uncertainty in Data-Driven Kalman Filtering for Partially Known State-Space Models](https://mlsanity.com/paper/2110.04738/) (2021)
- [Conditional Measurement Density Estimation in Sequential Monte Carlo via Normalizing Flow](https://mlsanity.com/paper/2203.08617/) (2022)
- [Interpretable Deep Feature Propagation for Early Action Recognition](https://mlsanity.com/paper/2107.05122/) (2021)
- [Recurrent Neural Networks and Universal Approximation of Bayesian Filters](https://mlsanity.com/paper/2211.00335/) (2022)
- [PVT++: A Simple End-to-End Latency-Aware Visual Tracking Framework](https://mlsanity.com/paper/2211.11629/) (2022)
- [Stacked Residuals of Dynamic Layers for Time Series Anomaly Detection](https://mlsanity.com/paper/2202.12457/) (2022)
- [Learning Sequential Latent Variable Models from Multimodal Time Series Data](https://mlsanity.com/paper/2204.10419/) (2022)
- [Streaming Motion Forecasting for Autonomous Driving](https://mlsanity.com/paper/2310.01351/) (2023)
- [Self-Supervised Inference in State-Space Models](https://mlsanity.com/paper/2107.13349/) (2021)
- [f-Cal: Calibrated aleatoric uncertainty estimation from neural networks for robot perception](https://mlsanity.com/paper/2109.13913/) (2021)
- [FILT3R: Latent State Adaptive Kalman Filter for Streaming 3D Reconstruction](https://mlsanity.com/paper/2603.18493/) (2026)
- [DiffPF: Differentiable Particle Filtering with Generative Sampling via Conditional Diffusion Models](https://mlsanity.com/paper/2507.15716/) (2025)

## Abstract (arXiv)

Generative state estimators based on probabilistic filters and smoothers are one of the most popular classes of state estimators for robots and autonomous vehicles. However, generative models have limited capacity to handle rich sensory observations, such as camera images, since they must model the entire distribution over sensor readings. Discriminative models do not suffer from this limitation, but are typically more complex to train as latent variable models for state estimation. We present an alternative approach where the parameters of the latent state distribution are directly optimized as a deterministic computation graph, resulting in a simple and effective gradient descent algorithm for training discriminative state estimators. We show that this procedure can be used to train state estimators that use complex input, such as raw camera images, which must be processed using expressive nonlinear function approximators such as convolutional neural networks. Our model can be viewed as a type of recurrent neural network, and the connection to probabilistic filtering allows us to design a network architecture that is particularly well suited for state estimation. We evaluate our approach on synthetic tracking task with raw image inputs and on the visual odometry task in the KITTI dataset. The results show significant improvement over both standard generative approaches and regular recurrent neural networks.