mlsanityData to 14 Oct 2026

Semantics-aware LiDAR-Only Pseudo Point Cloud Generation for 3D Object Detection

arXiv 2309.08932 ↗First posted 16 Sep 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Although LiDAR sensors are crucial for autonomous systems due to providing precise depth information, they struggle with capturing fine object details, especially at a distance, due to sparse and non-uniform data. Recent advances introduced pseudo-LiDAR, i.e., synthetic dense point clouds, using additional modalities such as cameras to enhance 3D object detection. We present a novel LiDAR-only framework that augments raw scans with denser pseudo point clouds by solely relying on LiDAR sensors and scene semantics, omitting the need for cameras. Our framework first utilizes a segmentation model to extract scene semantics from raw point clouds, and then employs a multi-modal domain translator to generate synthetic image segments and depth cues without real cameras. This yields a dense pseudo point cloud enriched with semantic information. We also introduce a new semantically guided projection method, which enhances detection performance by retaining only relevant pseudo points. We applied our framework to different advanced 3D object detection methods and reported up to 2.9% performance upgrade. We also obtained comparable results on the KITTI 3D object detection dataset, in contrast to other state-of-the-art LiDAR-only detectors.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 38 references that are in the map. First posted on arXiv 16 Sep 2023.

Builds on

33 of 38 references.

References · 33

  1. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  2. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  3. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  4. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  5. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  6. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  7. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  8. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  9. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  10. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  11. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  12. 2020Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detectioncited by 410
21 more
  1. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 368
  2. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  3. 2020SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Drivingcited by 226
  4. 2019Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Drivingcited by 188
  5. 2021Multimodal Virtual Point 3D Detectioncited by 166
  6. 2020EPNet: Enhancing Point Features with Image Semantics for 3D Object Detectioncited by 164
  7. 2022Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Cloudscited by 131
  8. 2019IoU Loss for 2D/3D Object Detectioncited by 128
  9. 2021SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloudcited by 119
  10. 2020Rethinking Pseudo-LiDAR Representationcited by 109
  11. 2021Improving 3D Object Detection with Channel-wise Transformercited by 108
  12. 2021FusionPainting: Multimodal Fusion with Adaptive Attention for 3D Object Detectioncited by 92
  13. 2022Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completioncited by 82
  14. 2023Virtual Sparse Convolution for Multimodal 3D Object Detectioncited by 81
  15. 2021Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detectioncited by 79
  16. 2021Behind the Curtain: Learning Occluded Shapes for 3D Object Detectioncited by 58
  17. 2022Graph R-CNN: Towards Accurate 3D Object Detection with Semantic-Decorated Local Graphcited by 42
  18. 2019SDNet: Semantically Guided Depth Estimation Networkcited by 11
  19. 2021Semantics-aware Multi-modal Domain Translation:From LiDAR Point Clouds to Panoramic Color Imagescited by 6
  20. 2019FusionMapping: Learning Depth Prediction with Monocular Images and 2D Laser Scanscited by 1
  21. 2023Depth- and Semantics-aware Multi-modal Domain Translation: Generating 3D Panoramic Color Images from LiDAR Point Cloudscited by 1

Built on by

Not cited yet.