# Learning to Segment 3D Point Clouds in 2D Image Space

arXiv: https://arxiv.org/abs/2003.05593 · HTML: https://mlsanity.com/paper/2003.05593/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Mar 2020.

## Built on by (most cited first, 24 of 24)

- [Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR-based Perception](https://mlsanity.com/paper/2109.05441/) (2021)
- [STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset](https://mlsanity.com/paper/2203.09065/) (2022)
- [SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Clouds](https://mlsanity.com/paper/2201.04494/) (2022)
- [Clustering based Point Cloud Representation Learning for 3D Analysis](https://mlsanity.com/paper/2307.14605/) (2023)
- [Voint Cloud: Multi-View Point Cloud Representation for 3D Understanding](https://mlsanity.com/paper/2111.15363/) (2021)
- [Robust Structured Declarative Classifiers for 3D Point Clouds: Defending Adversarial Attacks with Implicit Gradients](https://mlsanity.com/paper/2203.15245/) (2022)
- [Position-Guided Point Cloud Panoptic Segmentation Transformer](https://mlsanity.com/paper/2303.13509/) (2023)
- [From SLAM to Situational Awareness: Challenges and Survey](https://mlsanity.com/paper/2110.00273/) (2021)
- [PnP-3D: A Plug-and-Play for 3D Point Clouds](https://mlsanity.com/paper/2108.07378/) (2021)
- [Exploiting Local Geometry for Feature and Graph Construction for Better 3D Point Cloud Processing with Graph Neural Networks](https://mlsanity.com/paper/2103.15226/) (2021)
- [Theoretical bounds on data requirements for the ray-based classification](https://mlsanity.com/paper/2103.09577/) (2021)
- [Medial Spectral Coordinates for 3D Shape Analysis](https://mlsanity.com/paper/2111.13295/) (2021)
- [Graph Classification via Discriminative Edge Feature Learning](https://mlsanity.com/paper/2210.02060/) (2022)
- [Robust 3D Point Clouds Classification based on Declarative Defenders](https://mlsanity.com/paper/2410.09691/) (2024)
- [EllipsoidNet: Ellipsoid Representation for Point Cloud Classification and Segmentation](https://mlsanity.com/paper/2103.02517/) (2021)
- [Revisiting 2D Convolutional Neural Networks for Graph-based Applications](https://mlsanity.com/paper/2105.11016/) (2021)
- [Unsupervised Contrastive Learning with Simple Transformation for 3D Point Cloud Data](https://mlsanity.com/paper/2110.06632/) (2021)
- [Hypergraph Convolutional Network based Weakly Supervised Point Cloud Semantic Segmentation with Scene-Level Annotations](https://mlsanity.com/paper/2211.01174/) (2022)
- [Human Semantic Segmentation using Millimeter-Wave Radar Sparse Point Clouds](https://mlsanity.com/paper/2304.14132/) (2023)
- [TS40K: a 3D Point Cloud Dataset of Rural Terrain and Electrical Transmission System](https://mlsanity.com/paper/2405.13989/) (2024)
- [Enhancing Power Grid Inspections with Machine Learning](https://mlsanity.com/paper/2502.13037/) (2025)
- [Automating Infrastructure Surveying: A Framework for Geometric Measurements and Compliance Assessment Using Point Cloud Data](https://mlsanity.com/paper/2505.05752/) (2025)
- [InfraDiffusion: zero-shot depth map restoration with diffusion models and prompted segmentation from sparse infrastructure point clouds](https://mlsanity.com/paper/2509.03324/) (2025)
- [GIBLy: Improving 3D Semantic Segmentation through an Architecture-Agnostic Lightweight Geometric Inductive Bias Layer](https://mlsanity.com/paper/2605.24243/) (2026)

## Abstract (arXiv)

In contrast to the literature where local patterns in 3D point clouds are captured by customized convolutional operators, in this paper we study the problem of how to effectively and efficiently project such point clouds into a 2D image space so that traditional 2D convolutional neural networks (CNNs) such as U-Net can be applied for segmentation. To this end, we are motivated by graph drawing and reformulate it as an integer programming problem to learn the topology-preserving graph-to-grid mapping for each individual point cloud. To accelerate the computation in practice, we further propose a novel hierarchical approximate algorithm. With the help of the Delaunay triangulation for graph construction from point clouds and a multi-scale U-Net for segmentation, we manage to demonstrate the state-of-the-art performance on ShapeNet and PartNet, respectively, with significant improvement over the literature. Code is available at https://github.com/Zhang-VISLab.