# PI-RCNN: An Efficient Multi-sensor 3D Object Detector with Point-based Attentive Cont-conv Fusion Module

arXiv: https://arxiv.org/abs/1911.06084 · HTML: https://mlsanity.com/paper/1911.06084/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2019.

## Built on by (most cited first, 30 of 56)

- [TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers](https://mlsanity.com/paper/2203.11496/) (2022)
- [3D Object Detection for Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2206.09474/) (2022)
- [SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud](https://mlsanity.com/paper/2104.09804/) (2021)
- [Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion](https://mlsanity.com/paper/2203.09780/) (2022)
- [LoGoNet: Towards Accurate 3D Object Detection with Local-to-Global Cross-Modal Fusion](https://mlsanity.com/paper/2303.03595/) (2023)
- [Multi-modal Sensor Fusion for Auto Driving Perception: A Survey](https://mlsanity.com/paper/2202.02703/) (2022)
- [AutoAlign: Pixel-Instance Feature Aggregation for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2201.06493/) (2022)
- [MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2303.08600/) (2023)
- [Multi-Modal 3D Object Detection in Autonomous Driving: a Survey](https://mlsanity.com/paper/2106.12735/) (2021)
- [EPNet++: Cascade Bi-directional Fusion for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2112.11088/) (2021)
- [VPFNet: Improving 3D Object Detection with Virtual Point based LiDAR and Stereo Data Fusion](https://mlsanity.com/paper/2111.14382/) (2021)
- [Bridging the View Disparity Between Radar and Camera Features for Multi-modal Fusion 3D Object Detection](https://mlsanity.com/paper/2208.12079/) (2022)
- [Graph R-CNN: Towards Accurate 3D Object Detection with Semantic-Decorated Local Graph](https://mlsanity.com/paper/2208.03624/) (2022)
- [MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment](https://mlsanity.com/paper/2209.00244/) (2022)
- [STCrowd: A Multimodal Dataset for Pedestrian Perception in Crowded Scenes](https://mlsanity.com/paper/2204.01026/) (2022)
- [LIF-Seg: LiDAR and Camera Image Fusion for 3D LiDAR Semantic Segmentation](https://mlsanity.com/paper/2108.07511/) (2021)
- [GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2310.08261/) (2023)
- [CAT-Det: Contrastively Augmented Transformer for Multi-modal 3D Object Detection](https://mlsanity.com/paper/2204.00325/) (2022)
- [HRFuser: A Multi-resolution Sensor Fusion Architecture for 2D Object Detection](https://mlsanity.com/paper/2206.15157/) (2022)
- [Homogeneous Multi-modal Feature Fusion and Interaction for 3D Object Detection](https://mlsanity.com/paper/2210.09615/) (2022)
- [From Voxel to Point: IoU-guided 3D Object Detection for Point Cloud with Voxel-to-Point Decoder](https://mlsanity.com/paper/2108.03648/) (2021)
- [SuperPC: A Single Diffusion Model for Point Cloud Completion, Upsampling, Denoising, and Colorization](https://mlsanity.com/paper/2503.14558/) (2025)
- [DetMatch: Two Teachers are Better Than One for Joint 2D and 3D Semi-Supervised Object Detection](https://mlsanity.com/paper/2203.09510/) (2022)
- [LMM-PCQA: Assisting Point Cloud Quality Assessment with LMM](https://mlsanity.com/paper/2404.18203/) (2024)
- [MultiCorrupt: A Multi-Modal Robustness Dataset and Benchmark of LiDAR-Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2402.11677/) (2024)
- [FusionRCNN: LiDAR-Camera Fusion for Two-stage 3D Object Detection](https://mlsanity.com/paper/2209.10733/) (2022)
- [VoxelNextFusion: A Simple, Unified and Effective Voxel Fusion Framework for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2401.02702/) (2024)
- [Anchor-free 3D Single Stage Detector with Mask-Guided Attention for Point Cloud](https://mlsanity.com/paper/2108.03634/) (2021)
- [Recent Advances in Multi-modal 3D Intelligence: A Comprehensive Survey and Evaluation](https://mlsanity.com/paper/2310.15676/) (2023)
- [TransCAR: Transformer-based Camera-And-Radar Fusion for 3D Object Detection](https://mlsanity.com/paper/2305.00397/) (2023)

## Abstract (arXiv)

LIDAR point clouds and RGB-images are both extremely essential for 3D object detection. So many state-of-the-art 3D detection algorithms dedicate in fusing these two types of data effectively. However, their fusion methods based on Birds Eye View (BEV) or voxel format are not accurate. In this paper, we propose a novel fusion approach named Point-based Attentive Cont-conv Fusion(PACF) module, which fuses multi-sensor features directly on 3D points. Except for continuous convolution, we additionally add a Point-Pooling and an Attentive Aggregation to make the fused features more expressive. Moreover, based on the PACF module, we propose a 3D multi-sensor multi-task network called Pointcloud-Image RCNN(PI-RCNN as brief), which handles the image segmentation and 3D object detection tasks. PI-RCNN employs a segmentation sub-network to extract full-resolution semantic feature maps from images and then fuses the multi-sensor features via powerful PACF module. Beneficial from the effectiveness of the PACF module and the expressive semantic features from the segmentation module, PI-RCNN can improve much in 3D object detection. We demonstrate the effectiveness of the PACF module and PI-RCNN on the KITTI 3D Detection benchmark, and our method can achieve state-of-the-art on the metric of 3D AP.