mlsanityData to 7 Oct 2026

3D Video Object Detection with Learnable Object-Centric Global Optimization

arXiv 2303.15416 ↗First posted 27 Mar 2023cs.CVcited by 7

Not on the map. Why?

Abstract (arXiv)

We explore long-term temporal visual correspondence-based optimization for 3D video object detection in this work. Visual correspondence refers to one-to-one mappings for pixels across multiple images. Correspondence-based optimization is the cornerstone for 3D scene reconstruction but is less studied in 3D video object detection, because moving objects violate multi-view geometry constraints and are treated as outliers during scene reconstruction. We address this issue by treating objects as first-class citizens during correspondence-based optimization. In this work, we propose BA-Det, an end-to-end optimizable object detector with object-centric temporal correspondence learning and featuremetric object bundle adjustment. Empirically, we verify the effectiveness and efficiency of BA-Det for multiple baseline 3D detectors under various setups. Our BA-Det achieves SOTA performance on the large-scale Waymo Open Dataset (WOD) with only marginal computation cost. Our code is available at https://github.com/jiaweihe1996/BA-Det.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 43 of its 56 references that are in the map. First posted on arXiv 27 Mar 2023.

Builds on

43 of 56 references.

References · 43

  1. 2017Mask R-CNNcited by 7,259
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  4. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  5. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  6. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  7. 2015ORB-SLAM: a Versatile and Accurate Monocular SLAM Systemcited by 983
  8. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  9. 2017Deep Layer Aggregationcited by 436
  10. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  11. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  12. 2021Categorical Depth Distribution Network for Monocular 3D Object Detectioncited by 326
31 more
  1. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 251
  2. 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 199
  3. 2021Embracing Single Stride 3D Object Detector with Sparse Transformercited by 193
  4. 2021Objects are Different: Flexible Monocular 3D Object Detectioncited by 184
  5. 2018BA-Net: Dense Bundle Adjustment Networkcited by 173
  6. 2021Geometry Uncertainty Projection Network for Monocular 3D Object Detectioncited by 155
  7. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  8. 2020Kinematic 3D Object Detection in Monocular Videocited by 116
  9. 2020Rethinking Pseudo-LiDAR Representationcited by 109
  10. 2018CubeSLAM: Monocular 3D Object SLAMcited by 101
  11. 2020DSGN: Deep Stereo Geometry Network for 3D Object Detectioncited by 100
  12. 2018QuadricSLAM: Dual Quadrics from Object Detections as Landmarks in Object-oriented SLAMcited by 86
  13. 2021LIGA-Stereo: Learning LiDAR Geometry Aware Representations for Stereo-based 3D Detectorcited by 69
  14. 2016T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videoscited by 67
  15. 2022MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detectioncited by 65
  16. 2020AB3DMOT: A Baseline for 3D Multi-Object Tracking and New Evaluation Metricscited by 63
  17. 2022STS: Surround-view Temporal Stereo for Multi-view 3D Detectioncited by 53
  18. 2021Progressive Coordinate Transforms for Monocular 3D Object Detectioncited by 52
  19. 2022DEVIANT: Depth EquiVarIAnt NeTwork for Monocular 3D Object Detectioncited by 48
  20. 2021Immortal Tracker: Tracklet Never Diescited by 42
  21. 2022DID-M3D: Decoupling Instance Depth for Monocular 3D Object Detectioncited by 39
  22. 2022Monocular 3D Object Detection with Depth from Motioncited by 38
  23. 2023Super Sparse 3D Object Detectioncited by 32
  24. 2022MonoJSG: Joint Semantic and Geometric Cost Volume for Monocular 3D Object Detectioncited by 30
  25. 2022MV-FCOS3D++: Multi-View Camera-Only 4D Object Detection with Pretrained Monocular Backbonescited by 29
  26. 2022LET-3D-AP: Longitudinal Error Tolerant 3D Average Precision for Camera-Only 3D Detectioncited by 27
  27. 2022Densely Constrained Depth Estimator for Monocular 3D Object Detectioncited by 27
  28. 2018Stereo Vision-based Semantic 3D Object and Ego-motion Tracking for Autonomous Drivingcited by 26
  29. 2020Multi-View Optimization of Local Feature Geometrycited by 24
  30. 2022Hindsight is 20/20: Leveraging Past Traversals to Aid 3D Perceptioncited by 20
  31. 2021Neural Reprojection Error: Merging Feature Learning and Camera Pose Estimationcited by 17

Built on by

Cited by year

Citing papers per year: 2023: 5; 2024: 0; 2025: 2.

7 papers.

  1. 2023FSD V2: Improving Fully Sparse 3D Object Detection with Virtual Voxelscited by 34
  2. 2023Fully Sparse Fusion for 3D Object Detectioncited by 29
  3. 2023Tracking Objects with 3D Representation from Videoscited by 2
  4. 2023Weakly Supervised 3D Object Detection with Multi-Stage Generalizationcited by 2
  5. 2023DecoderTracker: Decoder-Only Method for Multiple-Object Trackingcited by 2
  6. 2025Few-Shot Learning in Video and 3D Object Detection: A Surveycited by 2
  7. 2025DetAny4D: Detect Anything 4D Temporally in a Streaming RGB Videocited by 0