# Cross-Modality 3D Object Detection

arXiv: https://arxiv.org/abs/2008.10436 · HTML: https://mlsanity.com/paper/2008.10436/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Aug 2020.

## Built on by (most cited first, 12 of 12)

- [3D Object Detection for Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2206.09474/) (2022)
- [GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2310.08261/) (2023)
- [MultiCorrupt: A Multi-Modal Robustness Dataset and Benchmark of LiDAR-Camera Fusion for 3D Object Detection](https://mlsanity.com/paper/2402.11677/) (2024)
- [DualCross: Cross-Modality Cross-Domain Adaptation for Monocular BEV Perception](https://mlsanity.com/paper/2305.03724/) (2023)
- [PAI3D: Painting Adaptive Instance-Prior for 3D Object Detection](https://mlsanity.com/paper/2211.08055/) (2022)
- [Toward Edge-Efficient Dense Predictions with Synergistic Multi-Task Neural Architecture Search](https://mlsanity.com/paper/2210.01384/) (2022)
- [PointSee: Image Enhances Point Cloud](https://mlsanity.com/paper/2211.01664/) (2022)
- [MARS: Multimodal Active Robotic Sensing for Articulated Characterization](https://mlsanity.com/paper/2407.01191/) (2024)
- [A Survey of Multi-sensor Fusion Perception for Embodied AI: Background, Methods, Challenges and Prospects](https://mlsanity.com/paper/2506.19769/) (2025)
- [GATE3D: Generalized Attention-based Task-synergized Estimation in 3D*](https://mlsanity.com/paper/2504.11014/) (2025)
- [Multi-Echo LiDAR for 3D Object Detection](https://mlsanity.com/paper/2107.11470/) (2021)
- [From One to Many: Dynamic Cross Attention Networks for LiDAR and Camera Fusion](https://mlsanity.com/paper/2209.12254/) (2022)

## Abstract (arXiv)

In this paper, we focus on exploring the fusion of images and point clouds for 3D object detection in view of the complementary nature of the two modalities, i.e., images possess more semantic information while point clouds specialize in distance sensing. To this end, we present a novel two-stage multi-modal fusion network for 3D object detection, taking both binocular images and raw point clouds as input. The whole architecture facilitates two-stage fusion. The first stage aims at producing 3D proposals through sparse point-wise feature fusion. Within the first stage, we further exploit a joint anchor mechanism that enables the network to utilize 2D-3D classification and regression simultaneously for better proposal generation. The second stage works on the 2D and 3D proposal regions and fuses their dense features. In addition, we propose to use pseudo LiDAR points from stereo matching as a data augmentation method to densify the LiDAR points, as we observe that objects missed by the detection network mostly have too few points especially for far-away objects. Our experiments on the KITTI dataset show that the proposed multi-stage fusion helps the network to learn better representations.