# Online Multi-Camera 3D Tracking via ID Prediction over Recurrent Sparse Queries

arXiv: https://arxiv.org/abs/2609.18363 · HTML: https://mlsanity.com/paper/2609.18363/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 31 references that are in the map.
- First posted on arXiv 16 Sep 2026.

## Builds on (most cited first, 27 of 27)

- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking](https://mlsanity.com/paper/2203.14360/) (2022)
- [Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusion](https://mlsanity.com/paper/2211.10581/) (2022)
- [MeMOT: Multi-Object Tracking with Memory](https://mlsanity.com/paper/2203.16761/) (2022)
- [Sparse4D v2: Recurrent Temporal Fusion with Sparse Model](https://mlsanity.com/paper/2305.14018/) (2023)
- [MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queries](https://mlsanity.com/paper/2205.00613/) (2022)
- [Multiview Detection with Feature Perspective Transformation](https://mlsanity.com/paper/2007.07247/) (2020)
- [Sparse4D v3: Advancing End-to-End 3D Detection and Tracking](https://mlsanity.com/paper/2311.11722/) (2023)
- [Multiple Object Tracking as ID Prediction](https://mlsanity.com/paper/2403.16848/) (2024)
- [Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Tracking](https://mlsanity.com/paper/2302.03802/) (2023)
- [Multiview Detection with Shadow Transformer (and View-Coherent Data Augmentation)](https://mlsanity.com/paper/2108.05888/) (2021)
- [LMGP: Lifted Multicut Meets Geometry Projections for Multi-Camera Multi-Object Tracking](https://mlsanity.com/paper/2111.11892/) (2021)
- [EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye View](https://mlsanity.com/paper/2310.13350/) (2023)
- [ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association](https://mlsanity.com/paper/2405.08909/) (2024)
- [Lifting Multi-View Detection and Tracking to the Bird's Eye View](https://mlsanity.com/paper/2403.12573/) (2024)
- [ReST: A Reconfigurable Spatial-Temporal Graph Model for Multi-Camera Multi-Object Tracking](https://mlsanity.com/paper/2308.13229/) (2023)
- [The 9th AI City Challenge](https://mlsanity.com/paper/2508.13564/) (2025)
- [MCBLT: Multi-Camera Multi-Object 3D Tracking in Long Videos](https://mlsanity.com/paper/2412.00692/) (2024)
- [The 10th AI City Challenge](https://mlsanity.com/paper/2608.17044/) (2026)
- [GMT: Effective Global Framework for Multi-Camera Multi-Target Tracking](https://mlsanity.com/paper/2407.01007/) (2024)
- [A Unified 3D Object Perception Framework for Real-Time Outside-In Multi-Camera Systems](https://mlsanity.com/paper/2601.10819/) (2026)

## Abstract (arXiv)

Online multi camera 3D tracking must maintain scene global identities across synchronized views, yet query-based trackers carry these identities only implicitly in the instance bank, where they fragment upon query interruption. We present an online architecture that recovers association accuracy by predicting IDs explicitly over recurrent sparse queries. An outside-in Sparse4D detector fuses calibrated views into world frame 3D detections while propagating a sparse query bank, and a causal MOTIP ID decoder associates detections against a finite trajectory memory. We adapt MOTIP's relative-ID prediction and recycled slot runtime to globally fused 3D observations, and introduce metric spatial gating and proximity based newborn recovery. On the official 2026 AI City Challenge Track 1 test set, our method raises HOTA from 29.63 with native instance bank identities to 38.01, primarily through an AssA increase from 20.83 to 31.10, and ranks third on the public leaderboard. Full-sequence validation over all 9,000 frames of each scene shows that decoupled ID training improves HOTA over native identities, whereas continuing detector training alongside the detached ID objective produces scene-dependent gains and losses.