# Latent-KalmanNet: Learned Kalman Filtering for Tracking from High-Dimensional Signals

arXiv: https://arxiv.org/abs/2304.07827 · HTML: https://mlsanity.com/paper/2304.07827/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2023.

## Built on by (most cited first, 17 of 17)

- [GSP-KalmanNet: Tracking Graph Signals via Neural-Aided Kalman Filtering](https://mlsanity.com/paper/2311.16602/) (2023)
- [AI-Aided Kalman Filters](https://mlsanity.com/paper/2410.12289/) (2024)
- [Deep Variational Sequential Monte Carlo for High-Dimensional Observations](https://mlsanity.com/paper/2501.05982/) (2025)
- [Near Field Localization via AI-Aided Subspace Methods](https://mlsanity.com/paper/2504.00599/) (2025)
- [DBPnet: Damper Characteristics-Based Bayesian Physics-Informed Neural Network for Wheel Load Estimation](https://mlsanity.com/paper/2605.24860/) (2026)
- [Learning Flock: Enhancing Sets of Particles for Multi~Sub-State Particle Filtering with Neural Augmentation](https://mlsanity.com/paper/2408.11348/) (2024)
- [Energy-Efficient State Estimation with 1-Bit Sensing: A Bussgang-Kalman Framework for Internet of Things](https://mlsanity.com/paper/2507.17284/) (2025)
- [Sun sensor calibration algorithms: A systematic mapping and survey](https://mlsanity.com/paper/2507.21541/) (2025)
- [Motion Estimation for Multi-Object Tracking using KalmanNet with Semantic-Independent Encoding](https://mlsanity.com/paper/2509.11323/) (2025)
- [Q-Net: Queue Length Estimation via Kalman-based Neural Networks](https://mlsanity.com/paper/2509.24725/) (2025)
- [pDANSE: Particle-based Data-driven Nonlinear State Estimation from Nonlinear Measurements](https://mlsanity.com/paper/2510.27503/) (2025)
- [Filtering Jump Markov Systems with Partially Known Dynamics: A Model-Based Deep Learning Approach](https://mlsanity.com/paper/2511.09569/) (2025)
- [VSE: Variational state estimation of complex model-free process](https://mlsanity.com/paper/2601.21887/) (2026)
- [Learning to Distributedly Estimate under Partially Known Dynamics: A Covariance-Agnostic Neural Kalman Consensus Filter](https://mlsanity.com/paper/2606.28441/) (2026)
- [Hierarchical Variational Kalman Filtering](https://mlsanity.com/paper/2607.00877/) (2026)
- [Edge-Aware Thermal Infrared UAV Swarm Tracking](https://mlsanity.com/paper/2607.12544/) (2026)
- [Unscented KalmanNet: Structure-Preserving Deep Learning with Calibrated Posterior Uncertainty under Incomplete Physics and Unknown Noise](https://mlsanity.com/paper/2608.04201/) (2026)

## Abstract (arXiv)

The Kalman filter (KF) is a widely-used algorithm for tracking dynamic systems that are captured by state space (SS) models. The need to fully describe a SS model limits its applicability under complex settings, e.g., when tracking based on visual data, and the processing of high-dimensional signals often induces notable latency. These challenges can be treated by mapping the measurements into latent features obeying some postulated closed-form SS model, and applying the KF in the latent space. However, the validity of this approximated SS model may constitute a limiting factor. In this work, we study tracking from high-dimensional measurements under complex settings using a hybrid model-based/data-driven approach. By gradually tackling the challenges in handling the observations model and the task, we develop Latent-KalmanNet, which implements tracking from high-dimensional measurements by leveraging data to jointly learn the KF along with the latent space mapping. Latent-KalmanNet combines a learned encoder with data-driven tracking in the latent space using the recently proposed-KalmanNet, while identifying the ability of each of these trainable modules to assist its counterpart via providing a suitable prior (by KalmanNet) and by learning a latent representation that facilitates data-aided tracking (by the encoder). Our empirical results demonstrate that the proposed Latent-KalmanNet achieves improved accuracy and run-time performance over both model-based and data-driven techniques by learning a surrogate latent representation that most facilitates tracking, while operating with limited complexity and latency.