# Sensor Fusion for Joint 3D Object Detection and Semantic Segmentation

arXiv: https://arxiv.org/abs/1904.11466 · HTML: https://mlsanity.com/paper/1904.11466/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2019.

## Built on by (most cited first, 30 of 48)

- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers](https://mlsanity.com/paper/2203.11496/) (2022)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds](https://mlsanity.com/paper/2207.04397/) (2022)
- [Offboard 3D Object Detection from Point Cloud Sequences](https://mlsanity.com/paper/2103.05073/) (2021)
- [3D Object Detection for Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2206.09474/) (2022)
- [Multi-modal Sensor Fusion for Auto Driving Perception: A Survey](https://mlsanity.com/paper/2202.02703/) (2022)
- [MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2204.12667/) (2022)
- [EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentation](https://mlsanity.com/paper/2106.15277/) (2021)
- [Cross-modal Learning for Domain Adaptation in 3D Semantic Segmentation](https://mlsanity.com/paper/2101.07253/) (2021)
- [HRFuser: A Multi-resolution Sensor Fusion Architecture for 2D Object Detection](https://mlsanity.com/paper/2206.15157/) (2022)
- [Deep Learning Based 3D Segmentation: A Survey](https://mlsanity.com/paper/2103.05423/) (2021)
- [Complementing Onboard Sensors with Satellite Map: A New Perspective for HD Map Construction](https://mlsanity.com/paper/2308.15427/) (2023)
- [Homogeneous Multi-modal Feature Fusion and Interaction for 3D Object Detection](https://mlsanity.com/paper/2210.09615/) (2022)
- [Let Images Give You More:Point Cloud Cross-Modal Training for Shape Analysis](https://mlsanity.com/paper/2210.04208/) (2022)
- [Multimodality Helps Few-shot 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2410.22489/) (2024)
- [MVFuseNet: Improving End-to-End Object Detection and Motion Forecasting through Multi-View Fusion of LiDAR Data](https://mlsanity.com/paper/2104.10772/) (2021)
- [Revisiting 3D Object Detection From an Egocentric Perspective](https://mlsanity.com/paper/2112.07787/) (2021)
- [Learning to Simulate Realistic LiDARs](https://mlsanity.com/paper/2209.10986/) (2022)
- [Image Understands Point Cloud: Weakly Supervised 3D Semantic Segmentation via Association Learning](https://mlsanity.com/paper/2209.07774/) (2022)
- [FusionRCNN: LiDAR-Camera Fusion for Two-stage 3D Object Detection](https://mlsanity.com/paper/2209.10733/) (2022)
- [Far3Det: Towards Far-Field 3D Detection](https://mlsanity.com/paper/2211.13858/) (2022)
- [M2DA: Multi-Modal Fusion Transformer Incorporating Driver Attention for Autonomous Driving](https://mlsanity.com/paper/2403.12552/) (2024)
- [Robust 3D Semantic Occupancy Prediction with Calibration-free Spatial Transformation](https://mlsanity.com/paper/2411.12177/) (2024)
- [VIN: Voxel-based Implicit Network for Joint 3D Object Detection and Segmentation for Lidars](https://mlsanity.com/paper/2107.02980/) (2021)
- [Real-Time Multi-Modal Semantic Fusion on Unmanned Aerial Vehicles with Label Propagation for Cross-Domain Adaptation](https://mlsanity.com/paper/2210.09739/) (2022)
- [Towards Single Stage Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2106.10309/) (2021)
- [A Survey of Multi-sensor Fusion Perception for Embodied AI: Background, Methods, Challenges and Prospects](https://mlsanity.com/paper/2506.19769/) (2025)
- [Investigating the Effect of Sensor Modalities in Multi-Sensor Detection-Prediction Models](https://mlsanity.com/paper/2101.03279/) (2021)

## Abstract (arXiv)

In this paper, we present an extension to LaserNet, an efficient and state-of-the-art LiDAR based 3D object detector. We propose a method for fusing image data with the LiDAR data and show that this sensor fusion method improves the detection performance of the model especially at long ranges. The addition of image data is straightforward and does not require image labels. Furthermore, we expand the capabilities of the model to perform 3D semantic segmentation in addition to 3D object detection. On a large benchmark dataset, we demonstrate our approach achieves state-of-the-art performance on both object detection and semantic segmentation while maintaining a low runtime.