# Track Initialization and Re-Identification for~3D Multi-View Multi-Object Tracking

arXiv: https://arxiv.org/abs/2405.18606 · HTML: https://mlsanity.com/paper/2405.18606/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 68 references that are in the map.
- First posted on arXiv 28 May 2024.

## Builds on (most cited first, 30 of 31)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019)
- [Rethinking the competition between detection and ReID in Multi-Object Tracking](https://mlsanity.com/paper/2010.12138/) (2020)
- [Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification](https://mlsanity.com/paper/1809.04427/) (2018)
- [MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queries](https://mlsanity.com/paper/2205.00613/) (2022)
- [POI: Multiple Object Tracking with High Performance Detection and Appearance Feature](https://mlsanity.com/paper/1610.06136/) (2016)
- [Deep Cosine Metric Learning for Person Re-Identification](https://mlsanity.com/paper/1812.00442/) (2018)
- [Multiview Detection with Feature Perspective Transformation](https://mlsanity.com/paper/2007.07247/) (2020)
- [Robust Object Detection under Occlusion with Context-Aware CompositionalNets](https://mlsanity.com/paper/2005.11643/) (2020)
- [Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking](https://mlsanity.com/paper/2302.03802/) (2023)
- [Deep Occlusion Reasoning for Multi-Camera Multi-Target Detection](https://mlsanity.com/paper/1704.05775/) (2017)
- [Deep Multi-camera People Detection](https://mlsanity.com/paper/1702.04593/) (2017)
- [Cross-View Cross-Scene Multi-View Crowd Counting](https://mlsanity.com/paper/2205.01551/) (2022)
- [Robust Instance Segmentation through Reasoning about Multi-Object Occlusion](https://mlsanity.com/paper/2012.02107/) (2020)
- [3D Random Occlusion and Multi-Layer Projection for Deep Multi-Camera Pedestrian Localization](https://mlsanity.com/paper/2207.10895/) (2022)
- [How Trustworthy are Performance Evaluations for Basic Vision Tasks?](https://mlsanity.com/paper/2008.03533/) (2020)
- [Online Visual Multi-Object Tracking via Labeled Random Finite Set Filtering](https://mlsanity.com/paper/1611.06011/) (2016)
- [Tracking Cells and their Lineages via Labeled Random Finite Sets](https://mlsanity.com/paper/2104.10964/) (2021)
- [Multi-sensor Joint Adaptive Birth Sampler for Labeled Random Finite Set Tracking](https://mlsanity.com/paper/2109.04355/) (2021)
- [Urban Traffic Surveillance (UTS): A fully probabilistic 3D tracking approach based on 2D detections](https://mlsanity.com/paper/2105.14993/) (2021)
- [An Overview of Particle Methods for Random Finite Set Models](https://mlsanity.com/paper/1602.03945/) (2016)
- [Robust multi-sensor Generalized Labeled Multi-Bernoulli filter](https://mlsanity.com/paper/2106.00208/) (2021)

## Built on by (most cited first, 7 of 7)

- [Cognitive Disentanglement for Referring Multi-Object Tracking](https://mlsanity.com/paper/2503.11496/) (2025)
- [3D Multi-Object Tracking Employing MS-GLMB Filter for Autonomous Driving](https://mlsanity.com/paper/2410.14977/) (2024)
- [Hybrid Attention for Robust RGB-T Pedestrian Detection in Real-World Conditions](https://mlsanity.com/paper/2411.03576/) (2024)
- [SWIR-LightFusion: Multi-spectral Semantic Fusion of Synthetic SWIR with Thermal IR (LWIR/MWIR) and RGB](https://mlsanity.com/paper/2510.13404/) (2025)
- [3D Ground Truth Reconstruction from Multi-Camera Annotations Using UKF](https://mlsanity.com/paper/2511.17609/) (2025)
- [Efficient Online 3D Multi-Camera Multi-Object Tracking and Pose Estimation](https://mlsanity.com/paper/2604.16522/) (2026)
- [Beyond the Survey: A Systematic Empirical Study of Detection and Association in Visual MOT](https://mlsanity.com/paper/2609.22291/) (2026)

## Abstract (arXiv)

We propose a 3D multi-object tracking (MOT) solution using only 2D detections from monocular cameras, which automatically initiates/terminates tracks as well as resolves track appearance-reappearance and occlusions. Moreover, this approach does not require detector retraining when cameras are reconfigured but only the camera matrices of reconfigured cameras need to be updated. Our approach is based on a Bayesian multi-object formulation that integrates track initiation/termination, re-identification, occlusion handling, and data association into a single Bayes filtering recursion. However, the exact filter that utilizes all these functionalities is numerically intractable due to the exponentially growing number of terms in the (multi-object) filtering density, while existing approximations trade-off some of these functionalities for speed. To this end, we develop a more efficient approximation suitable for online MOT by incorporating object features and kinematics into the measurement model, which improves data association and subsequently reduces the number of terms. Specifically, we exploit the 2D detections and extracted features from multiple cameras to provide a better approximation of the multi-object filtering density to realize the track initiation/termination and re-identification functionalities. Further, incorporating a tractable geometric occlusion model based on 2D projections of 3D objects on the camera planes realizes the occlusion handling functionality of the filter. Evaluation of the proposed solution on challenging datasets demonstrates significant improvements and robustness when camera configurations change on-the-fly, compared to existing multi-view MOT solutions. The source code is publicly available at https://github.com/linh-gist/mv-glmb-ab.