# PG-RCNN: Semantic Surface Point Generation for 3D Object Detection

arXiv: https://arxiv.org/abs/2307.12637 · HTML: https://mlsanity.com/paper/2307.12637/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 45 references that are in the map.
- First posted on arXiv 24 Jul 2023.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [PCN: Point Completion Network](https://mlsanity.com/paper/1808.00671/) (2018)
- [From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network](https://mlsanity.com/paper/1907.03670/) (2019)
- [Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud](https://mlsanity.com/paper/2003.01251/) (2020)
- [STD: Sparse-to-Dense 3D Object Detector for Point Cloud](https://mlsanity.com/paper/1907.10471/) (2019)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [3D Object Detection with Pointformer](https://mlsanity.com/paper/2012.11409/) (2020)
- [Multimodal Virtual Point 3D Detection](https://mlsanity.com/paper/2111.06881/) (2021)
- [LiDAR R-CNN: An Efficient and Universal 3D Object Detector](https://mlsanity.com/paper/2103.15297/) (2021)
- [Fast Point R-CNN](https://mlsanity.com/paper/1908.02990/) (2019)
- [Improving 3D Object Detection with Channel-wise Transformer](https://mlsanity.com/paper/2108.10723/) (2021)
- [SPG: Unsupervised Domain Adaptation for 3D Object Detection via Semantic Point Generation](https://mlsanity.com/paper/2108.06709/) (2021)
- [Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion](https://mlsanity.com/paper/2203.09780/) (2022)
- [Point Density-Aware Voxels for LiDAR 3D Object Detection](https://mlsanity.com/paper/2203.05662/) (2022)
- [Behind the Curtain: Learning Occluded Shapes for 3D Object Detection](https://mlsanity.com/paper/2112.02205/) (2021)
- [VPFNet: Improving 3D Object Detection with Virtual Point based LiDAR and Stereo Data Fusion](https://mlsanity.com/paper/2111.14382/) (2021)
- [F-LOAM: Fast LiDAR Odometry And Mapping](https://mlsanity.com/paper/2107.00822/) (2021)

## Built on by (most cited first, 12 of 12)

- [Weakly Supervised 3D Object Detection via Multi-Level Visual Guidance](https://mlsanity.com/paper/2312.07530/) (2023)
- [SparseDet: A Simple and Effective Framework for Fully Sparse LiDAR-based 3D Object Detection](https://mlsanity.com/paper/2406.10907/) (2024)
- [TripleMixer: A 3D Point Cloud Denoising Model for Adverse Weather](https://mlsanity.com/paper/2408.13802/) (2024)
- [Leveraging Anchor-based LiDAR 3D Object Detection via Point Assisted Sample Selection](https://mlsanity.com/paper/2403.01978/) (2024)
- [PVAFN: Point-Voxel Attention Fusion Network with Multi-Pooling Enhancing for 3D Object Detection](https://mlsanity.com/paper/2408.14600/) (2024)
- [FSMDet: Vision-guided feature diffusion for fully sparse 3D detector](https://mlsanity.com/paper/2409.06945/) (2024)
- [Pillar-Voxel Fusion Network for 3D Object Detection in Airborne Hyperspectral Point Clouds](https://mlsanity.com/paper/2504.09506/) (2025)
- [Enhancing LiDAR Point Features with Foundation Model Priors for 3D Object Detection](https://mlsanity.com/paper/2507.13899/) (2025)
- [Image-Guided Semantic Pseudo-LiDAR Point Generation for 3D Object Detection](https://mlsanity.com/paper/2409.14985/) (2024)
- [Intrinsic-feature-guided 3D Object Detection](https://mlsanity.com/paper/2504.00382/) (2025)
- [3DPillars: Pillar-based two-stage 3D object detection](https://mlsanity.com/paper/2509.05780/) (2025)
- [TransBridge: Boost 3D Object Detection by Scene-Level Completion with Transformer Decoder](https://mlsanity.com/paper/2512.11926/) (2025)

## Abstract (arXiv)

One of the main challenges in LiDAR-based 3D object detection is that the sensors often fail to capture the complete spatial information about the objects due to long distance and occlusion. Two-stage detectors with point cloud completion approaches tackle this problem by adding more points to the regions of interest (RoIs) with a pre-trained network. However, these methods generate dense point clouds of objects for all region proposals, assuming that objects always exist in the RoIs. This leads to the indiscriminate point generation for incorrect proposals as well. Motivated by this, we propose Point Generation R-CNN (PG-RCNN), a novel end-to-end detector that generates semantic surface points of foreground objects for accurate detection. Our method uses a jointly trained RoI point generation module to process the contextual information of RoIs and estimate the complete shape and displacement of foreground objects. For every generated point, PG-RCNN assigns a semantic feature that indicates the estimated foreground probability. Extensive experiments show that the point clouds generated by our method provide geometrically and semantically rich information for refining false positive and misaligned proposals. PG-RCNN achieves competitive performance on the KITTI benchmark, with significantly fewer parameters than state-of-the-art models. The code is available at https://github.com/quotation2520/PG-RCNN.