mlsanityData to 7 Oct 2026

RGB and LiDAR fusion based 3D Semantic Segmentation for Autonomous Driving

arXiv 1906.00208 ↗First posted 1 Jun 2019cs.CV cs.AI cs.LGcited by 39

Not on the map. Why?

Abstract (arXiv)

LiDAR has become a standard sensor for autonomous driving applications as they provide highly precise 3D point clouds. LiDAR is also robust for low-light scenarios at night-time or due to shadows where the performance of cameras is degraded. LiDAR perception is gradually becoming mature for algorithms including object detection and SLAM. However, semantic segmentation algorithm remains to be relatively less explored. Motivated by the fact that semantic segmentation is a mature algorithm on image data, we explore sensor fusion based 3D segmentation. Our main contribution is to convert the RGB image to a polar-grid mapping representation used for LiDAR and design early and mid-level fusion architectures. Additionally, we design a hybrid fusion architecture that combines both fusion algorithms. We evaluate our algorithm on KITTI dataset which provides segmentation annotation for cars, pedestrians and cyclists. We evaluate two state-of-the-art architectures namely SqueezeSeg and PointSeg and improve the mIoU score by 10 % in both cases relative to the LiDAR only baseline.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 6; 2023: 10; 2024: 10; 2025: 6; 2026: 1.

39 papers.

Sort:

By area:

  1. 2022V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformercited by 320
  2. 20222DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Cloudscited by 157
  3. 2022MM-TTA: Multi-Modal Test-Time Adaptation for 3D Semantic Segmentationcited by 76
  4. 2023MSeg3D: Multi-modal 3D Semantic Segmentation for Autonomous Drivingcited by 68
  5. 2021EPMF: Efficient Perception-aware Multi-sensor Fusion for 3D Semantic Segmentationcited by 59
  6. 2023UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebasecited by 57
  7. 2023Spatio-Temporal Domain Awareness for Multi-Agent Collaborative Perceptioncited by 52
  8. 2021Spatio-Contextual Deep Network Based Multimodal Pedestrian Detection For Autonomous Drivingcited by 25
  9. 2022Let Images Give You More:Point Cloud Cross-Modal Training for Shape Analysiscited by 24
  10. 2024Multimodality Helps Few-shot 3D Point Cloud Semantic Segmentationcited by 20
  11. 2021BEVDetNet: Bird's Eye View LiDAR Point Cloud based Real-time 3D Object Detection for Autonomous Drivingcited by 17
  12. 2021LiMoSeg: Real-time Bird's Eye View based LiDAR Motion Segmentationcited by 15
27 more
  1. 2022Image Understands Point Cloud: Weakly Supervised 3D Semantic Segmentation via Association Learningcited by 15
  2. 2024TASeg: Temporal Aggregation Network for LiDAR Semantic Segmentationcited by 13
  3. 2023Revisiting Modality Imbalance In Multimodal Pedestrian Detectioncited by 10
  4. 2025MemorySAM: Memorize Modalities and Semantics with Segment Anything Model 2 for Multi-modal Semantic Segmentationcited by 9
  5. 2024CLFT: Camera-LiDAR Fusion Transformer for Semantic Segmentation in Autonomous Drivingcited by 7
  6. 2024Robust 3D Semantic Occupancy Prediction with Calibration-free Spatial Transformationcited by 7
  7. 2023CMDFusion: Bidirectional Fusion Network with Cross-modality Knowledge Distillation for LIDAR Semantic Segmentationcited by 6
  8. 2024MambaST: A Plug-and-Play Cross-Spectral Spatial-Temporal Fuser for Efficient Pedestrian Detectioncited by 6
  9. 2024ZOPP: A Framework of Zero-shot Offboard Panoptic Perception for Autonomous Drivingcited by 6
  10. 2025CAML: Collaborative Auxiliary Modality Learning for Multi-Agent Systemscited by 6
  11. 2022Semantic Segmentation of Fruits on Multi-sensor Fused Data in Natural Orchardscited by 4
  12. 2024Fusion-then-Distillation: Toward Cross-modal Positive Distillation for Domain Adaptive 3D Semantic Segmentationcited by 4
  13. 2023NLOS Dies Twice: Challenges and Solutions of V2X for Cooperative Perceptioncited by 3
  14. 2021Fast and Accurate: Video Enhancement using Sparse Depthcited by 2
  15. 2023Revisiting Multi-modal 3D Semantic Segmentation in Real-world Autonomous Drivingcited by 2
  16. 2025MMCD: Multi-Modal Collaborative Decision-Making for Connected Autonomy with Knowledge Distillationcited by 2
  17. 2023Review of Deep Reinforcement Learning for Autonomous Drivingcited by 1
  18. 2023A survey on deep learning approaches for data integration in autonomous driving systemcited by 1
  19. 2025EGFormer: Towards Efficient and Generalizable Multimodal Semantic Segmentationcited by 1
  20. 2021An Evaluation of RGB and LiDAR Fusion for Semantic Segmentationcited by 0
  21. 2023An Efficient Plane Extraction Approach for Bundle Adjustment on LiDAR Point cloudscited by 0
  22. 2024ELiTe: Efficient Image-to-LiDAR Knowledge Transfer for Semantic Segmentationcited by 0
  23. 2024Enhancing Generalizability of Representation Learning for Data-Efficient 3D Scene Understandingcited by 0
  24. 2024Uplifting Range-View-based 3D Semantic Segmentation in Real-Time with Multi-Sensor Fusioncited by 0
  25. 2025Multimodal Point Cloud Semantic Segmentation With Virtual Point Enhancementcited by 0
  26. 2025Domain Adaptation-Based Crossmodal Knowledge Distillation for 3D Semantic Segmentationcited by 0
  27. 2026Contextual Range-View Projection for 3D LiDAR Point Cloudscited by 0