mlsanityData to 7 Oct 2026

MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queries

arXiv 2205.00613 ↗First posted 2 May 2022cs.CV cs.AIcited by 90

Not on the map. Why?

Abstract (arXiv)

Accurate and consistent 3D tracking from multiple cameras is a key component in a vision-based autonomous driving system. It involves modeling 3D dynamic objects in complex scenes across multiple cameras. This problem is inherently challenging due to depth estimation, visual occlusions, appearance ambiguity, etc. Moreover, objects are not consistently associated across time and cameras. To address that, we propose an end-to-end MUlti-camera TRacking framework called MUTR3D. In contrast to prior works, MUTR3D does not explicitly rely on the spatial and appearance similarity of objects. Instead, our method introduces textit3D track query to model spatial and appearance coherent track for each object that appears in multiple cameras and multiple frames. We use camera transformations to link 3D trackers with their observations in 2D images. Each tracker is further refined according to the features that are obtained from camera images. MUTR3D uses a set-to-set loss to measure the difference between the predicted tracking results and the ground truths. Therefore, it does not require any post-processing such as non-maximum suppression and/or bounding box association. MUTR3D outperforms state-of-the-art methods by 5.3 AMOTA on the nuScenes dataset. Code is available at: urlhttps://github.com/a1600012888/MUTR3D.

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). Builds on 43 of its 50 references that are in the map. First posted on arXiv 2 May 2022.

Builds on

43 of 50 references.

References · 43

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  2. 2020End-to-End Object Detection with Transformerscited by 7,384
  3. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  6. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  7. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  8. 2017Deformable Convolutional Networkscited by 1,481
  9. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  10. 2019Objects as Pointscited by 1,172
  11. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  12. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
31 more
  1. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 908
  2. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  3. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  4. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 515
  5. 2020Tracking Objects as Pointscited by 483
  6. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  7. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  8. 2020End-to-End Video Instance Segmentation with Transformerscited by 443
  9. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  10. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 379
  11. 2020TransTrack: Multiple Object Tracking with Transformercited by 370
  12. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
  13. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
  14. 2019Disentangling Monocular 3D Object Detectioncited by 215
  15. 2019Learning a Neural Solver for Multiple Object Trackingcited by 198
  16. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 190
  17. 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
  18. 2018ROI-10D: Monocular Lifting of 2D Detection to 6D Pose and Metric Shapecited by 123
  19. 2021Learning to Track with Object Permanencecited by 117
  20. 2020GNN3DMOT: Graph Neural Network for 3D Multi-Object Tracking with Multi-Feature Learningcited by 113
  21. 2020Rethinking Pseudo-LiDAR Representationcited by 109
  22. 2021SimpleTrack: Understanding and Rethinking 3D Multi-object Trackingcited by 102
  23. 2021Monocular Quasi-Dense 3D Object Trackingcited by 80
  24. 2020Probabilistic 3D Multi-Modal, Multi-Object Tracking for Autonomous Drivingcited by 73
  25. 2015DenseBox: Unifying Landmark Localization with End to End Object Detectioncited by 58
  26. 2021DEFT: Detection Embeddings for Trackingcited by 56
  27. 2021Learnable Online Graph Representations for 3D Multi-Object Trackingcited by 38
  28. 2018Mono-Camera 3D Multi-Object Tracking Using Deep Learning Detections and PMBM Filteringcited by 32
  29. 2021Score refinement for confidence-based 3D multi-object trackingcited by 32
  30. 2019Monocular 3D Object Detection via Geometric Reasoning on Keypointscited by 31
  31. 2019FANTrack: 3D Multi-Object Tracking with Feature Association Networkcited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 21; 2024: 26; 2025: 23; 2026: 14.

90 papers.

Sort:

By area:

  1. 2022Planning-oriented Autonomous Drivingcited by 1,068
  2. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  3. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 239
  4. 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 165
  5. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  6. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 125
  7. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  8. 2023Visual Point Cloud Forecasting enables Scalable Autonomous Drivingcited by 97
  9. 2023ADAPT: Action-aware Driving Caption Transformercited by 78
  10. 2023DrivingDiffusion: Layout-Guided multi-view driving scene video generation with latent diffusion modelcited by 75
  11. 2023SSCBench: A Large-Scale 3D Semantic Scene Completion Benchmark for Autonomous Drivingcited by 72
  12. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 69
38 more
  1. 2023Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perceptioncited by 65
  2. 2023Standing Between Past and Future: Spatio-Temporal Modeling for Multi-Camera 3D Multi-Object Trackingcited by 54
  3. 2024HIMap: HybrId Representation Learning for End-to-end Vectorized HD Map Constructioncited by 48
  4. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 46
  5. 2024ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Associationcited by 33
  6. 2022CC-3DT: Panoramic 3D Object Tracking via Cross-Camera Fusioncited by 31
  7. 2024GraphAD: Interaction Scene Graph for End-to-end Autonomous Drivingcited by 25
  8. 2023QUEST: Query Stream for Practical Cooperative Perceptioncited by 23
  9. 2023Towards Motion Forecasting with Real-World Perception Inputs: Are End-to-End Approaches Competitive?cited by 22
  10. 2023ByteTrackV2: 2D and 3D Multi-Object Tracking by Associating Every Detection Boxcited by 20
  11. 2023MotionTrack: End-to-End Transformer-based Multi-Object Tracing with LiDAR-Camera Fusioncited by 19
  12. 2023From Semi-supervised to Omni-supervised Room Layout Estimation Using Point Cloudscited by 18
  13. 2022Quality Matters: Embracing Quality Clues for Robust 3D Multi-Object Trackingcited by 17
  14. 2024DualAD: Disentangling the Dynamic and Static World for End-to-End Drivingcited by 17
  15. 2023S.T.A.R.-Track: Latent Motion Models for End-to-End 3D Object Tracking with Adaptive Spatio-Temporal Appearance Representationscited by 12
  16. 2025CoopTrack: Exploring End-to-End Learning for Efficient Cooperative Sequential Perceptioncited by 12
  17. 2023DORT: Modeling Dynamic Objects in Recurrent for Multi-Camera 3D Object Detection and Trackingcited by 11
  18. 2023Delving into Motion-Aware Matching for Monocular 3D Object Trackingcited by 11
  19. 2024PIP-Net: Pedestrian Intention Prediction in the Wildcited by 11
  20. 2024Fast-Poly: A Fast Polyhedral Framework For 3D Multi-Object Trackingcited by 11
  21. 2024Seeing Beyond Views: Multi-View Driving Scene Video Generation with Holistic Attentioncited by 10
  22. 2025CoopDETR: A Unified Cooperative Perception Framework for 3D Detection via Object Querycited by 9
  23. 2025Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Trackingcited by 9
  24. 2023Transformer-Based Autonomous Driving Models and Deployment-Oriented Compression: A Surveycited by 8
  25. 2024MoST: Multi-modality Scene Tokenization for Motion Predictioncited by 8
  26. 2024MCTR: Multi Camera Tracking Transformercited by 8
  27. 2025Self-Supervised Sparse Sensor Fusion for Long Range Perceptioncited by 8
  28. 2024Track Initialization and Re-Identification for~3D Multi-View Multi-Object Trackingcited by 7
  29. 2023Predict to Detect: Prediction-guided 3D Object Detection using Sequential Imagescited by 6
  30. 2024WidthFormer: Toward Efficient Transformer-based BEV View Transformationcited by 5
  31. 2024Valeo4Cast: A Modular Approach to End-to-End Forecastingcited by 5
  32. 2024Panacea+: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 5
  33. 2025Divide and Merge: Motion and Semantic Learning in End-to-End Autonomous Drivingcited by 5
  34. 2025A Survey of Multi-sensor Fusion Perception for Embodied AI: Background, Methods, Challenges and Prospectscited by 5
  35. 2026TruckDrive: Long-Range Autonomous Highway Driving Datasetcited by 5
  36. 2023So you think you can track?cited by 4
  37. 2024A Survey of Vision Transformers in Autonomous Driving: Current Trends and Future Directionscited by 4
  38. 2024JDT3D: Addressing the Gaps in LiDAR-Based Tracking-by-Attentioncited by 4

50 most cited of 90 All 90 (JSON)