mlsanityData to 7 Oct 2026

DORT: Modeling Dynamic Objects in Recurrent for Multi-Camera 3D Object Detection and Tracking

arXiv 2303.16628 ↗First posted 29 Mar 2023cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

Recent multi-camera 3D object detectors usually leverage temporal information to construct multi-view stereo that alleviates the ill-posed depth estimation. However, they typically assume all the objects are static and directly aggregate features across frames. This work begins with a theoretical and empirical analysis to reveal that ignoring the motion of moving objects can result in serious localization bias. Therefore, we propose to model Dynamic Objects in RecurrenT (DORT) to tackle this problem. In contrast to previous global Bird-Eye-View (BEV) methods, DORT extracts object-wise local volumes for motion estimation that also alleviates the heavy computational burden. By iteratively refining the estimated object motion and location, the preceding features can be precisely aggregated to the current frame to mitigate the aforementioned adverse effects. The simple framework has two significant appealing properties. It is flexible and practical that can be plugged into most camera-based 3D object detectors. As there are predictions of object motion in the loop, it can easily track objects across frames according to their nearest center distances. Without bells and whistles, DORT outperforms all the previous methods on the nuScenes detection and tracking benchmarks with 62.5% NDS and 57.6% AMOTA, respectively. The source code will be released.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 57 of its 61 references that are in the map. First posted on arXiv 29 Mar 2023.

Builds on

57 of 61 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  3. 2017Mask R-CNNcited by 7,259
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2022A ConvNet for the 2020scited by 3,545
  6. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  7. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  8. 2019Objects as Pointscited by 1,172
  9. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  10. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  11. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  12. 2016Simple Online and Realtime Trackingcited by 862
38 more
  1. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  2. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  3. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 655
  4. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  5. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  6. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 490
  7. 2020Tracking Objects as Pointscited by 483
  8. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 455
  9. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  10. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  11. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  12. 2019Recurrent MVSNet for High-resolution Multi-view Stereo Depth Inferencecited by 252
  13. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 251
  14. 2021Is Pseudo-Lidar needed for Monocular 3D Object detection?cited by 248
  15. 2020Quasi-Dense Similarity Learning for Multiple Object Trackingcited by 220
  16. 2022Unifying Voxel-based Representation with Transformer for 3D Object Detectioncited by 202
  17. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
  18. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 190
  19. 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
  20. 2021Objects are Different: Flexible Monocular 3D Object Detectioncited by 184
  21. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 177
  22. 2018BA-Net: Dense Bundle Adjustment Networkcited by 173
  23. 2020RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Drivingcited by 150
  24. 2021Delving into Localization Errors for Monocular 3D Object Detectioncited by 138
  25. 2020RAFT-3D: Scene Flow using Rigid-Motion Embeddingscited by 121
  26. 2020Kinematic 3D Object Detection in Monocular Videocited by 116
  27. 2021SimpleTrack: Understanding and Rethinking 3D Multi-object Trackingcited by 102
  28. 2022MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queriescited by 90
  29. 2022BEVStereo: Enhancing Depth Estimation in Multi-view 3D Object Detection with Dynamic Temporal Stereocited by 89
  30. 2021AutoShape: Real-Time Shape-Aware Monocular 3D Object Detectioncited by 87
  31. 2021Monocular Quasi-Dense 3D Object Trackingcited by 80
  32. 2021Monocular 3D Object Detection: An Extrinsic Parameter Free Approachcited by 69
  33. 2018MVDepthNet: Real-time Multiview Depth Estimation Neural Networkcited by 59
  34. 2022Polar Parametrization for Vision-based Surround-View 3D Detectioncited by 53
  35. 2020Disp R-CNN: Stereo 3D Object Detection via Shape Prior Guided Instance Disparity Estimationcited by 43
  36. 2022Learning Ego 3D Representation as Ray Tracingcited by 43
  37. 2022Monocular 3D Object Detection with Depth from Motioncited by 38
  38. 2022CC-3DT: Panoramic 3D Object Tracking via Cross-Camera Fusioncited by 31

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 6; 2025: 2; 2026: 1.

11 papers.

  1. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 69
  2. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 46
  3. 2023BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detectioncited by 35
  4. 2024DualAD: Disentangling the Dynamic and Static World for End-to-End Drivingcited by 17
  5. 2025Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Trackingcited by 9
  6. 2024TempBEV: Improving Learned BEV Encoders with Combined Image and BEV Space Temporal Aggregationcited by 5
  7. 2024RockTrack: A 3D Robust Multi-Camera-Ken Multi-Object Tracking Frameworkcited by 2
  8. 2024SynCL: A Synergistic Training Strategy with Instance-Aware Contrastive Learning for End-to-End Multi-Camera 3D Trackingcited by 2
  9. 2026Leveraging Previous-Traversal Point Cloud Map Priors for Camera-Based 3D Object Detection and Trackingcited by 2
  10. 2024Distribution Discrepancy and Feature Heterogeneity for Active 3D Object Detectioncited by 1
  11. 2025Towards Latency-Aware 3D Streaming Perception for Autonomous Drivingcited by 1