mlsanityData to 7 Oct 2026

SimPB++: Simultaneously Detecting 2D and 3D Objects from Multiple Cameras

arXiv 2605.01924 ↗First posted 3 May 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Simultaneous perception of 2D objects in perspective view and 3D objects in Bird's Eye View (BEV) is challenging for multi-camera autonomous driving. Existing two-stage pipelines use 2D results only as a one-time cue for 3D detection. We propose SimPB++, which simultaneously detects 2D objects in perspective and 3D objects in BEV from multiple cameras. It unifies both tasks into an end-to-end model with a hybrid decoder architecture, coupling multi-view 2D and 3D decoders interactively. Two novel modules enable deep interaction: Dynamic Query Allocation adaptively assigns 2D queries to 3D candidates, and Adaptive Query Aggregation refines 3D representations using multi-view 2D features, forming a cyclic 3D-2D-3D refinement. For multi-view 2D detection, we use Query-group Attention for intra-group communication. We also design a Crop-and-Scale strategy for long-range perception and a Propagating Denoising strategy with an auxiliary RoI detector. SimPB++ supports mixed supervision with 2D-only and fully annotated data, reducing reliance on expensive 3D labels. Experiments show state-of-the-art performance on nuScenes for both tasks and strong long-range detection (up to 150m) on Argoverse2.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 51 of its 59 references that are in the map. First posted on arXiv 3 May 2026.

Builds on

51 of 59 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2020End-to-End Object Detection with Transformerscited by 7,384
  5. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  7. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  8. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
  9. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  10. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  11. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  12. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
38 more
  1. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 711
  2. 2023Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecastingcited by 701
  3. 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
  4. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 569
  5. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 568
  6. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 550
  7. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 486
  8. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 444
  9. 2021Conditional DETR for Fast Training Convergencecited by 422
  10. 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 351
  11. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  12. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 340
  13. 2023Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detectioncited by 313
  14. 2022BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervisioncited by 302
  15. 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 277
  16. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 273
  17. 2019Libra R-CNN: Towards Balanced Learning for Object Detectioncited by 255
  18. 2019M3D-RPN: Monocular 3D Region Proposal Network for Object Detectioncited by 251
  19. 2022Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detectioncited by 177
  20. 2019An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detectioncited by 163
  21. 2022Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusioncited by 161
  22. 2023SparseBEV: High-Performance Sparse 3D Object Detection from Multi-Camera Videoscited by 149
  23. 2021Sparse DETR: Efficient End-to-End Object Detection with Learnable Sparsitycited by 98
  24. 2023Sparse4D v2: Recurrent Temporal Fusion with Sparse Modelcited by 92
  25. 2023Far3D: Expanding the Horizon for Surround-view 3D Object Detectioncited by 70
  26. 2023Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perceptioncited by 65
  27. 2023Temporal Enhanced Training of Multi-view 3D Object Detector via Historical Object Predictioncited by 49
  28. 2024Ray Denoising: Depth-aware Hard Negative Sampling for Multi-view 3D Object Detectioncited by 36
  29. 2023BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detectioncited by 35
  30. 2022DETR4D: Direct Multi-View 3D Object Detection with Sparse Attentioncited by 21
  31. 2022Focal-PETR: Embracing Foreground for Efficient Multi-Camera 3D Object Detectioncited by 19
  32. 2019Learning Object-specific Distance from a Monocular Imagecited by 15
  33. 2024GeoBEV: Learning Geometric BEV Representation for Multi-view 3D Object Detectioncited by 10
  34. 2024SimPB: A Single Model for 2D and 3D Object Detection from Multiple Camerascited by 9
  35. 2025Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Trackingcited by 9
  36. 2024SparseFusion: Efficient Sparse Multi-Modal Fusion Framework for Long-Range 3D Perceptioncited by 8
  37. 2024Improving Distant 3D Object Detection Using 2D Box Supervisioncited by 6
  38. 2023Towards Long-Range 3D Object Detection for Autonomous Vehiclescited by 5

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.