mlsanityData to 7 Oct 2026

PointSee: Image Enhances Point Cloud

arXiv 2211.01664 ↗First posted 3 Nov 2022cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

There is a trend to fuse multi-modal information for 3D object detection (3OD). However, the challenging problems of low lightweightness, poor flexibility of plug-and-play, and inaccurate alignment of features are still not well-solved, when designing multi-modal fusion newtorks. We propose PointSee, a lightweight, flexible and effective multi-modal fusion solution to facilitate various 3OD networks by semantic feature enhancement of LiDAR point clouds assembled with scene images. Beyond the existing wisdom of 3OD, PointSee consists of a hidden module (HM) and a seen module (SM): HM decorates LiDAR point clouds using 2D image information in an offline fusion manner, leading to minimal or even no adaptations of existing 3OD networks; SM further enriches the LiDAR point clouds by acquiring point-wise representative semantic features, leading to enhanced performance of existing 3OD networks. Besides the new architecture of PointSee, we propose a simple yet efficient training strategy, to ease the potential inaccurate regressions of 2D object detection networks. Extensive experiments on the popular outdoor/indoor benchmarks show numerical improvements of our PointSee over twenty-two state-of-the-arts.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 57 of its 78 references that are in the map. First posted on arXiv 3 Nov 2022.

Builds on

57 of 78 references.

References · 57

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  2. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  3. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  4. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  5. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  6. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  7. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  8. 2015Fast R-CNNcited by 2,930
  9. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  10. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
  11. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  12. 2019FCOS: Fully Convolutional One-Stage Object Detectioncited by 1,573
38 more
  1. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  2. 2019Objects as Pointscited by 1,172
  3. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  4. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  5. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  6. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
  7. 2018CornerNet: Detecting Objects as Paired Keypointscited by 763
  8. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  9. 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 719
  10. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  11. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  12. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  13. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  14. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 366
  15. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 325
  16. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  17. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  18. 2021Group-Free 3D Object Detection via Transformerscited by 215
  19. 20203D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detectioncited by 176
  20. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  21. 2020EPNet: Enhancing Point Features with Image Semantics for 3D Object Detectioncited by 164
  22. 2019MVX-Net: Multimodal VoxelNet for 3D Object Detectioncited by 162
  23. 2019Bottom-up Object Detection by Grouping Extreme and Center Pointscited by 156
  24. 2022Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Cloudscited by 131
  25. 2021LiDAR R-CNN: An Efficient and Universal 3D Object Detectorcited by 129
  26. 2019Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detectioncited by 124
  27. 2020H3DNet: 3D Object Detection Using Hybrid Geometric Primitivescited by 123
  28. 2019Fast Point R-CNNcited by 119
  29. 2020ImVoteNet: Boosting 3D Object Detection in Point Clouds with Image Votescited by 119
  30. 2015Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Imagescited by 111
  31. 2019TANet: Robust 3D Object Detection from Point Clouds with Triple Attentioncited by 108
  32. 2020CIA-SSD: Confident IoU-Aware Single-Stage Object Detector From Point Cloudcited by 107
  33. 20213D Object Detection for Autonomous Driving: A Surveycited by 106
  34. 2022Voxel Set Transformer: A Set-to-Set Approach to 3D Object Detection from Point Cloudscited by 102
  35. 2022AutoAlign: Pixel-Instance Feature Aggregation for Multi-Modal 3D Object Detectioncited by 70
  36. 2020Generative Sparse Detection Networks for 3D Single-shot Object Detectioncited by 69
  37. 2021Multi-Modal 3D Object Detection in Autonomous Driving: a Surveycited by 63
  38. 2022SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detectioncited by 52

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1; 2025: 2.

5 papers.

  1. 2023PointGame: Geometrically and Adaptively Masked Auto-Encoder on Point Cloudscited by 4
  2. 2023PointSmile: Point Self-supervised Learning via Curriculum Mutual Informationcited by 2
  3. 2025SIM-Net: A Multimodal Fusion Network Using Inferred 3D Object Shape Point Clouds from RGB Images for 2D Classificationcited by 2
  4. 2024PointCG: Self-supervised Point Cloud Learning via Joint Completion and Generationcited by 0
  5. 2025PVNet: Point-Voxel Interaction LiDAR Scene Upsampling Via Diffusion Modelscited by 0