mlsanityData to 14 Oct 2026

Online 3D Multi-Camera Perception through Robust 2D Tracking and Depth-based Late Aggregation

arXiv 2509.09946 ↗First posted 12 Sep 2025cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Multi-Target Multi-Camera Tracking (MTMC) is an essential computer vision task for automating large-scale surveillance. With camera calibration and depth information, the targets in the scene can be projected into 3D space, offering unparalleled levels of automatic perception of a 3D environment. However, tracking in the 3D space requires replacing all 2D tracking components from the ground up, which may be infeasible for existing MTMC systems. In this paper, we present an approach for extending any online 2D multi-camera tracking system into 3D space by utilizing depth information to reconstruct a target in point-cloud space, and recovering its 3D box through clustering and yaw refinement following tracking. We also introduced an enhanced online data association mechanism that leverages the target's local ID consistency to assign global IDs across frames. The proposed framework is evaluated on the 2025 AI City Challenge's 3D MTMC dataset, achieving 3rd place on the leaderboard.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 45 of its 70 references that are in the map. First posted on arXiv 12 Sep 2025.

Builds on

45 of 70 references.

References · 45

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2017Mask R-CNNcited by 7,277
  5. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  6. 2015Fast R-CNNcited by 2,933
  7. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  8. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  9. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  10. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  11. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 910
  12. 2016Simple Online and Realtime Trackingcited by 868
33 more
  1. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 823
  2. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 724
  3. 2020HOTA: A Higher Order Metric for Evaluating Multi-Object Trackingcited by 552
  4. 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 490
  5. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  6. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  7. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  8. 2022Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Trackingcited by 346
  9. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 327
  10. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 274
  11. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 252
  12. 2018CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmarkcited by 214
  13. 2020SMOKE: Single-Stage Monocular 3D Object Detection via Keypoint Estimationcited by 204
  14. 2015DeepStereo: Learning to Predict New Views from the World's Imagerycited by 185
  15. 2021Objects are Different: Flexible Monocular 3D Object Detectioncited by 185
  16. 2022CLIP-ReID: Exploiting Vision-Language Model for Image Re-Identification without Concrete Text Labelscited by 181
  17. 2023RTMPose: Real-Time Multi-Person Pose Estimation based on MMPosecited by 175
  18. 2019Stereo R-CNN based 3D Object Detection for Autonomous Drivingcited by 163
  19. 2021Geometry Uncertainty Projection Network for Monocular 3D Object Detectioncited by 156
  20. 2020MonoPair: Monocular 3D Object Detection Using Pairwise Spatial Relationshipscited by 153
  21. 2020RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Drivingcited by 150
  22. 2023Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identificationcited by 132
  23. 2020DSGN: Deep Stereo Geometry Network for 3D Object Detectioncited by 101
  24. 2022MonoDETR: Depth-guided Transformer for Monocular 3D Object Detectioncited by 86
  25. 2021LMGP: Lifted Multicut Meets Geometry Projections for Multi-Camera Multi-Object Trackingcited by 34
  26. 2023EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye Viewcited by 33
  27. 2023ReST: A Reconfigurable Spatial-Temporal Graph Model for Multi-Camera Multi-Object Trackingcited by 24
  28. 2022ObjectSeeker: Certifiably Robust Object Detection against Patch Hiding Attacks via Patch-agnostic Maskingcited by 22
  29. 2017Multi-Target, Multi-Camera Tracking by Hierarchical Clustering: Recent Progress on DukeMTMC Projectcited by 19
  30. 2024The 8th AI City Challengecited by 16
  31. 2025The 9th AI City Challengecited by 12
  32. 2023Enhancing Multi-Camera People Tracking with Anchor-Guided Clustering and Spatio-Temporal Consistency ID Re-Assignmentcited by 9
  33. 2024MonoDETRNext: Next-Generation Accurate and Efficient Monocular 3D Object Detectorcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026A Unified 3D Object Perception Framework for Real-Time Outside-In Multi-Camera Systemscited by 2
  2. 2026Physical AI Smart Spaces: A Large-Scale Benchmark for Multi-Camera 3D Perception in Smart Spacescited by 0