# Deep Projective 3D Semantic Segmentation

arXiv: https://arxiv.org/abs/1705.03428 · HTML: https://mlsanity.com/paper/1705.03428/

## Key facts

- Cited by 81 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2017.

## Built on by (most cited first, 30 of 81)

- [An End-to-End Transformer Model for 3D Object Detection](https://mlsanity.com/paper/2109.08141/) (2021)
- [PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds](https://mlsanity.com/paper/2103.14635/) (2021)
- [2DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Clouds](https://mlsanity.com/paper/2207.04397/) (2022)
- [Revisiting Point Cloud Shape Classification with a Simple and Effective Baseline](https://mlsanity.com/paper/2106.05304/) (2021)
- [Neural 3D Scene Reconstruction with the Manhattan-world Assumption](https://mlsanity.com/paper/2205.02836/) (2022)
- [NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenes](https://mlsanity.com/paper/2111.13260/) (2021)
- [Instance Segmentation in 3D Scenes using Semantic Superpoint Tree Networks](https://mlsanity.com/paper/2108.07478/) (2021)
- [Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion](https://mlsanity.com/paper/2103.07074/) (2021)
- [Learning 3D Semantic Segmentation with only 2D Image Supervision](https://mlsanity.com/paper/2110.11325/) (2021)
- [Semantics for Robotic Mapping, Perception and Interaction: A Survey](https://mlsanity.com/paper/2101.00443/) (2021)
- [Contrastive Boundary Learning for Point Cloud Segmentation](https://mlsanity.com/paper/2203.05272/) (2022)
- [Generative Zero-Shot Learning for Semantic Segmentation of 3D Point Clouds](https://mlsanity.com/paper/2108.06230/) (2021)
- [ReDAL: Region-based and Diversity-aware Active Learning for Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2107.11769/) (2021)
- [POP-3D: Open-Vocabulary 3D Occupancy Prediction from Images](https://mlsanity.com/paper/2401.09413/) (2024)
- [OA-CNNs: Omni-Adaptive Sparse CNNs for 3D Semantic Segmentation](https://mlsanity.com/paper/2403.14418/) (2024)
- [Image2Point: 3D Point-Cloud Understanding with 2D Image Pretrained Models](https://mlsanity.com/paper/2106.04180/) (2021)
- [MotionSC: Data Set and Network for Real-Time Semantic Mapping in Dynamic Environments](https://mlsanity.com/paper/2203.07060/) (2022)
- [Attention-based Point Cloud Edge Sampling](https://mlsanity.com/paper/2302.14673/) (2023)
- [PU-Transformer: Point Cloud Upsampling Transformer](https://mlsanity.com/paper/2111.12242/) (2021)
- [Deep Learning Based 3D Segmentation: A Survey](https://mlsanity.com/paper/2103.05423/) (2021)
- [DRINet: A Dual-Representation Iterative Learning Network for Point Cloud Segmentation](https://mlsanity.com/paper/2108.04023/) (2021)
- [Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly](https://mlsanity.com/paper/2301.05033/) (2023)
- [KPConvX: Modernizing Kernel Point Convolution with Kernel Attention](https://mlsanity.com/paper/2405.13194/) (2024)
- [FPS-Net: A Convolutional Fusion Network for Large-Scale LiDAR Point Cloud Segmentation](https://mlsanity.com/paper/2103.00738/) (2021)
- [Learning Inter-Superpoint Affinity for Weakly Supervised 3D Instance Segmentation](https://mlsanity.com/paper/2210.05534/) (2022)
- [All Points Matter: Entropy-Regularized Distribution Alignment for Weakly-supervised 3D Segmentation](https://mlsanity.com/paper/2305.15832/) (2023)
- [PnP-3D: A Plug-and-Play for 3D Point Clouds](https://mlsanity.com/paper/2108.07378/) (2021)
- [GeoSegNet: Point Cloud Semantic Segmentation via Geometric Encoder-Decoder Modeling](https://mlsanity.com/paper/2207.06766/) (2022)
- [A Review of Panoptic Segmentation for Mobile Mapping Point Clouds](https://mlsanity.com/paper/2304.13980/) (2023)
- [DLA-Net: Learning Dual Local Attention Features for Semantic Segmentation of Large-Scale Building Facade Point Clouds](https://mlsanity.com/paper/2106.00376/) (2021)

## Abstract (arXiv)

Semantic segmentation of 3D point clouds is a challenging problem with numerous real-world applications. While deep learning has revolutionized the field of image semantic segmentation, its impact on point cloud data has been limited so far. Recent attempts, based on 3D deep learning approaches (3D-CNNs), have achieved below-expected results. Such methods require voxelizations of the underlying point cloud data, leading to decreased spatial resolution and increased memory consumption. Additionally, 3D-CNNs greatly suffer from the limited availability of annotated datasets. In this paper, we propose an alternative framework that avoids the limitations of 3D-CNNs. Instead of directly solving the problem in 3D, we first project the point cloud onto a set of synthetic 2D-images. These images are then used as input to a 2D-CNN, designed for semantic segmentation. Finally, the obtained prediction scores are re-projected to the point cloud to obtain the segmentation results. We further investigate the impact of multiple modalities, such as color, depth and surface normals, in a multi-stream network architecture. Experiments are performed on the recent Semantic3D dataset. Our approach sets a new state-of-the-art by achieving a relative gain of 7.9 %, compared to the previous best approach.