mlsanityData to 14 Oct 2026

Motion Estimation for Multi-Object Tracking using KalmanNet with Semantic-Independent Encoding

arXiv 2509.11323 ↗First posted 14 Sep 2025cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Motion estimation is a crucial component in multi-object tracking (MOT). It predicts the trajectory of objects by analyzing the changes in their positions in consecutive frames of images, reducing tracking failures and identity switches. The Kalman filter (KF) based on the linear constant-velocity model is one of the most commonly used methods in MOT. However, it may yield unsatisfactory results when KF's parameters are mismatched and objects move in non-stationary. In this work, we utilize the learning-aided filter to handle the motion estimation of MOT. In particular, we propose a novel method named Semantic-Independent KalmanNet (SIKNet), which encodes the state vector (the input feature) using a Semantic-Independent Encoder (SIE) by two steps. First, the SIE uses a 1D convolution with a kernel size of 1, which convolves along the dimension of homogeneous-semantic elements across different state vectors to encode independent semantic information. Then it employs a fully-connected layer and a nonlinear activation layer to encode nonlinear and cross-dependency information between heterogeneous-semantic elements. To independently evaluate the performance of the motion estimation module in MOT, we constructed a large-scale semi-simulated dataset from several open-source MOT datasets. Experimental results demonstrate that the proposed SIKNet outperforms the traditional KF and achieves superior robustness and accuracy than existing learning-aided filters. The code is available at (https://github.com/SongJgit/filternet and https://github.com/SongJgit/TBDTracker).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 41 of its 64 references that are in the map. First posted on arXiv 14 Sep 2025.

Builds on

41 of 64 references.

References · 41

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  2. 2015Fast R-CNNcited by 2,933
  3. 2022A Time Series is Worth 64 Words: Long-term Forecasting with Transformerscited by 2,039
  4. 2018An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modelingcited by 1,476
  5. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,071
  6. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
  7. 2016Simple Online and Realtime Trackingcited by 868
  8. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
  9. 2020HOTA: A Higher Order Metric for Evaluating Multi-Object Trackingcited by 552
  10. 2016MOT16: A Benchmark for Multi-Object Trackingcited by 532
  11. 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 490
  12. 2020MOT20: A benchmark for multi object tracking in crowded scenescited by 357
29 more
  1. 2022Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Trackingcited by 346
  2. 2022BoT-SORT: Robust Associations Multi-Pedestrian Trackingcited by 320
  3. 2020Detection and Tracking Meet Drones Challengecited by 263
  4. 2020Quasi-Dense Similarity Learning for Multiple Object Trackingcited by 220
  5. 2021DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motioncited by 217
  6. 2022StrongSORT: Make DeepSORT Great Againcited by 213
  7. 2016Structured Inference Networks for Nonlinear State Space Modelscited by 182
  8. 2020Model-Based Deep Learningcited by 141
  9. 2021KalmanNet: Neural Network Aided Kalman Filtering for Partially Known Dynamicscited by 132
  10. 2017A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learningcited by 107
  11. 2019Deep Learning in Video Multi-Object Tracking: A Surveycited by 107
  12. 2024DUET: Dual Clustering Enhanced Multivariate Time Series Forecastingcited by 107
  13. 2019The H3D Dataset for Full-Surround 3D Multi-Object Detection and Tracking in Crowded Urban Scenescited by 80
  14. 2022SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videoscited by 80
  15. 2020Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Drivingcited by 73
  16. 2016T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videoscited by 67
  17. 2019Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spacescited by 67
  18. 2022Hard to Track Objects with Irregular Motions and Similar Appearances? Make It Easier by Buffering the Matching Spacecited by 66
  19. 2024DiffMOT: A Real-time Diffusion-based Multiple Object Tracker with Non-linear Predictioncited by 59
  20. 2022Model-Based Deep Learning: On the Intersection of Deep Learning and Optimizationcited by 50
  21. 2022GIAOTracker: A comprehensive framework for MCMOT with global information and optimizing strategies in VisDrone 2021cited by 46
  22. 2023UCMCTrack: Multi-Object Tracking with Uniform Camera Motion Compensationcited by 44
  23. 2020KFNet: Learning Temporal Camera Relocalization using Kalman Filteringcited by 43
  24. 2021Camera Calibration and Player Localization in SoccerNet-v2 and Investigation of their Representations for Action Spottingcited by 43
  25. 2015What makes for effective detection proposals?cited by 38
  26. 2023Latent-KalmanNet: Learned Kalman Filtering for Tracking from High-Dimensional Signalscited by 17
  27. 2019DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Predictioncited by 13
  28. 2023Bayesian KalmanNet: Quantifying Uncertainty in Deep Learning Augmented Kalman Filtercited by 10
  29. 2023GSP-KalmanNet: Tracking Graph Signals via Neural-Aided Kalman Filteringcited by 10

Built on by

Not cited yet.