mlsanityData to 14 Oct 2026

SimPB: A Single Model for 2D and 3D Object Detection from Multiple Cameras

arXiv 2403.10353 ↗First posted 15 Mar 2024cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

The field of autonomous driving has attracted considerable interest in approaches that directly infer 3D objects in the Bird's Eye View (BEV) from multiple cameras. Some attempts have also explored utilizing 2D detectors from single images to enhance the performance of 3D detection. However, these approaches rely on a two-stage process with separate detectors, where the 2D detection results are utilized only once for token selection or query initialization. In this paper, we present a single model termed SimPB, which simultaneously detects 2D objects in the perspective view and 3D objects in the BEV space from multiple cameras. To achieve this, we introduce a hybrid decoder consisting of several multi-view 2D decoder layers and several 3D decoder layers, specifically designed for their respective detection tasks. A Dynamic Query Allocation module and an Adaptive Query Aggregation module are proposed to continuously update and refine the interaction between 2D and 3D results, in a cyclic 3D-2D-3D manner. Additionally, Query-group Attention is utilized to strengthen the interaction among 2D queries within each camera group. In the experiments, we evaluate our method on the nuScenes dataset and demonstrate promising results for both 2D and 3D detection tasks. Our code is available at: https://github.com/nullmax-vision/SimPB.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 43 of its 48 references that are in the map. First posted on arXiv 15 Mar 2024.

Builds on

43 of 48 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  7. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,422
  8. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  9. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  10. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,276
  11. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
  12. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
31 more
  1. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  2. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  3. 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
  4. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 573
  5. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  6. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  7. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  8. 2021Conditional DETR for Fast Training Convergencecited by 424
  9. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 352
  10. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  11. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 341
  12. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 315
  13. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  14. 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 278
  15. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 274
  16. 2019Libra R-CNN: Towards Balanced Learning for Object Detectioncited by 255
  17. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 252
  18. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 178
  19. 2019An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detectioncited by 163
  20. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 162
  21. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 150
  22. 2021Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsitycited by 98
  23. 2023Sparse4D v2: Recurrent Temporal Fusion with Sparse Modelcited by 93
  24. 2023Far3D: Expanding the Horizon for Surround-view 3D Object Detectioncited by 70
  25. 2023Sparse4D v3: Advancing End-to-End 3D Detection and Trackingcited by 70
  26. 2023Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perceptioncited by 65
  27. 2023Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Predictioncited by 49
  28. 2023BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detectioncited by 35
  29. 2022DETR4D: Direct Multi-View 3D Object Detection with Sparse Attentioncited by 21
  30. 2022Focal-PETR: Embracing Foreground for Efficient Multi-Camera 3D Object Detectioncited by 19
  31. 2023QE-BEV: Query Evolution for Bird's Eye View Object Detection in Varied Contextscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 6.

9 papers.

  1. 2024SpaRC: Sparse Radar-Camera Fusion for 3D Object Detectioncited by 8
  2. 2025CHARM3R: Towards Unseen Camera Height Robust Monocular 3D Detectorcited by 6
  3. 2026Towards Intrinsic-Aware Monocular 3D Object Detectioncited by 4
  4. 2025Rethinking the Spatio-Temporal Alignment of End-to-End 3D Perceptioncited by 3
  5. 2026DSERT-RoLL: Robust Multi-Modal Perception for Diverse Driving Conditions with Stereo Event-RGB-Thermal Cameras, 4D Radar, and Dual-LiDARcited by 3
  6. 2026SimPB++: Simultaneously Detecting 2D and 3D Objects from Multiple Camerascited by 0
  7. 2026PixelPilot: Scalable Vision-Language-Action Models for End-to-End Autonomous Drivingcited by 0
  8. 2026Variational Inference for Bird's Eye View Segmentation in Autonomous Drivingcited by 0
  9. 2026MatchFusion: Explicit-Implicit Instance Matching for Spatio-Temporal Multimodal Autonomous Drivingcited by 0