mlsanityData to 14 Oct 2026

SemanticBEVFusion: Rethink LiDAR-Camera Fusion in Unified Bird's-Eye View Representation for 3D Object Detection

arXiv 2212.04675 ↗First posted 9 Dec 2022cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

LiDAR and camera are two essential sensors for 3D object detection in autonomous driving. LiDAR provides accurate and reliable 3D geometry information while the camera provides rich texture with color. Despite the increasing popularity of fusing these two complementary sensors, the challenge remains in how to effectively fuse 3D LiDAR point cloud with 2D camera images. Recent methods focus on point-level fusion which paints the LiDAR point cloud with camera features in the perspective view or bird's-eye view (BEV)-level fusion which unifies multi-modality features in the BEV representation. In this paper, we rethink these previous fusion strategies and analyze their information loss and influences on geometric and semantic features. We present SemanticBEVFusion to deeply fuse camera features with LiDAR features in a unified BEV representation while maintaining per-modality strengths for 3D object detection. Our method achieves state-of-the-art performance on the large-scale nuScenes dataset, especially for challenging distant objects. The code will be made publicly available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 44 references that are in the map. First posted on arXiv 9 Dec 2022.

Builds on

36 of 44 references.

References · 36

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  6. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  7. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,577
  8. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  9. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  10. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  11. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
  12. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
24 more
  1. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  2. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
  3. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  4. 2021DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queriescited by 658
  5. 2019CenterNet: Keypoint Triplets for Object Detectioncited by 654
  6. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  7. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  8. 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
  9. 2022TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformerscited by 466
  10. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  11. 2021FCOS3D: Fully Convolutional One-Stage Monocular 3D Object Detectioncited by 447
  12. 2017Deep Layer Aggregationcited by 436
  13. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  14. 2022BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detectioncited by 341
  15. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 336
  16. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  17. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 202
  18. 2019End-to-End Multi-View Fusion for 3D Object Detection in LiDAR Point Cloudscited by 171
  19. 2021Multimodal Virtual Point 3D Detectioncited by 166
  20. 2022M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representationcited by 156
  21. 2021Probabilistic two-stage detectioncited by 130
  22. 2014Learning Rich Features from RGB-D Images for Object Detection and Segmentationcited by 127
  23. 2022DeepInteraction: 3D Object Detection via Modality Interactioncited by 125
  24. 2021FusionPainting: Multimodal Fusion with Adaptive Attention for 3D Object Detectioncited by 92

Built on by

Not cited yet.