# ModTrack: Sensor-Agnostic Multi-View Tracking via Identity-Informed PHD Filtering with Covariance Propagation

arXiv: https://arxiv.org/abs/2603.15812 · HTML: https://mlsanity.com/paper/2603.15812/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 37 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Builds on (most cited first, 22 of 22)

- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [Person Transfer GAN to Bridge Domain Gap for Person Re-Identification](https://mlsanity.com/paper/1711.08565/) (2017)
- [Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification](https://mlsanity.com/paper/1809.04427/) (2018)
- [Learning Generalisable Omni-Scale Representations for Person Re-Identification](https://mlsanity.com/paper/1910.06827/) (2019)
- [Multiview Detection with Feature Perspective Transformation](https://mlsanity.com/paper/2007.07247/) (2020)
- [4D Panoptic LiDAR Segmentation](https://mlsanity.com/paper/2102.12472/) (2021)
- [RadarScenes: A Real-World Radar Point Cloud Data Set for Automotive Applications](https://mlsanity.com/paper/2104.02493/) (2021)
- [Torchreid: A Library for Deep Learning Person Re-Identification in Pytorch](https://mlsanity.com/paper/1910.10093/) (2019)
- [Generalized optimal sub-pattern assignment metric](https://mlsanity.com/paper/1601.05585/) (2016)
- [EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye View](https://mlsanity.com/paper/2310.13350/) (2023)
- [Lifting Multi-View Detection and Tracking to the Bird's Eye View](https://mlsanity.com/paper/2403.12573/) (2024)
- [ReST: A Reconfigurable Spatial-Temporal Graph Model for Multi-Camera Multi-Object Tracking](https://mlsanity.com/paper/2308.13229/) (2023)
- [Real-time 3D Deep Multi-Camera Tracking](https://mlsanity.com/paper/2003.11753/) (2020)
- [MCBLT: Multi-Camera Multi-Object 3D Tracking in Long Videos](https://mlsanity.com/paper/2412.00692/) (2024)
- [MVTrajecter: Multi-View Pedestrian Tracking with Trajectory Motion Cost and Trajectory Appearance Cost](https://mlsanity.com/paper/2509.01157/) (2025)
- [MIC-BEV: Multi-Infrastructure Camera Bird's-Eye-View Transformer with Relation-Aware Fusion for 3D Object Detection](https://mlsanity.com/paper/2510.24688/) (2025)
- [Attention-Aware Multi-View Pedestrian Tracking](https://mlsanity.com/paper/2504.03047/) (2025)
- [Radar Tracker: Moving Instance Tracking in Sparse and Noisy Radar Point Clouds](https://mlsanity.com/paper/2507.03441/) (2025)
- [One Graph to Track Them All: Dynamic GNNs for Single- and Multi-View Tracking](https://mlsanity.com/paper/2507.08494/) (2025)
- [ATLASFusion: Aggregation Tracking with Location-Aware Sparse Fusion for Robust Spatio-Temporal Multi-View Pedestrian Tracking](https://mlsanity.com/paper/2509.08421/) (2025)

## Abstract (arXiv)

Multi-View Multi-Object Tracking (MV-MOT) aims to localize and maintain consistent identities of objects observed by multiple sensors. This task is challenging, as viewpoint changes and occlusion disrupt identity consistency across views and time. Recent end-to-end approaches address this by jointly learning 2D Bird's Eye View (BEV) representations and identity associations, achieving high tracking accuracy. However, these methods offer no principled uncertainty accounting and remain tightly coupled to their training configuration, limiting generalization across sensor layouts, modalities, or datasets without retraining. We propose ModTrack, a modular MV-MOT system that matches end-to-end performance while providing cross-modal, sensor-agnostic generalization and traceable uncertainty. ModTrack confines learning methods to just the textitDetection and Feature Extraction stage of the MV-MOT pipeline, performing all fusion, association, and tracking with closed-form analytical methods. Our design reduces each sensor's output to calibrated position-covariance pairs (z, R); cross-view clustering and precision-weighted fusion then yield unified estimates (hatz, hatR) for identity assignment and temporal tracking. A feedback-coupled, identity-informed Gaussian Mixture Probability Hypothesis Density (GM-PHD) filter with HMM motion modes uses these fused estimates to maintain identities under missed detections and heavy occlusion. ModTrack achieves 95.5 IDF1 and 91.4 MOTA on textitWildTrack, surpassing all prior modular methods by over 21 points and rivaling the state-of-the-art end-to-end methods while providing deployment flexibility they cannot. Specifically, the same tracker core transfers unchanged to textitMultiviewX and textitRadarScenes, with only perception-module replacement required to extend to new domains and sensor modalities.