# Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges

arXiv: https://arxiv.org/abs/2009.03137 · HTML: https://mlsanity.com/paper/2009.03137/

## Key facts

- Cited by 80 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Sep 2020.

## Built on by (most cited first, 30 of 80)

- [V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving](https://mlsanity.com/paper/2202.08449/) (2022)
- [Segment Any Point Cloud Sequences by Distilling Vision Foundation Models](https://mlsanity.com/paper/2306.09347/) (2023)
- [OpenIns3D: Snap and Lookup for 3D Open-vocabulary Instance Segmentation](https://mlsanity.com/paper/2309.00616/) (2023)
- [S2Looking: A Satellite Side-Looking Dataset for Building Change Detection](https://mlsanity.com/paper/2107.09244/) (2021)
- [Multi-modal Sensor Fusion for Auto Driving Perception: A Survey](https://mlsanity.com/paper/2202.02703/) (2022)
- [SQN: Weakly-Supervised Semantic Segmentation of Large-Scale 3D Point Clouds](https://mlsanity.com/paper/2104.04891/) (2021)
- [STPLS3D: A Large-Scale Synthetic and Real Aerial Photogrammetry 3D Point Cloud Dataset](https://mlsanity.com/paper/2203.09065/) (2022)
- [Learning Semantic Segmentation of Large-Scale Point Clouds with Random Sampling](https://mlsanity.com/paper/2107.02389/) (2021)
- [SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Clouds](https://mlsanity.com/paper/2201.04494/) (2022)
- [FRNet: Frustum-Range Networks for Scalable LiDAR Segmentation](https://mlsanity.com/paper/2312.04484/) (2023)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [The Hessigheim 3D (H3D) Benchmark on Semantic Segmentation of High-Resolution 3D Point Clouds and Textured Meshes from UAV LiDAR and Multi-View-Stereo](https://mlsanity.com/paper/2102.05346/) (2021)
- [GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds](https://mlsanity.com/paper/2305.16404/) (2023)
- [4D Contrastive Superflows are Dense 3D Representation Learners](https://mlsanity.com/paper/2407.06190/) (2024)
- [SAGD: Boundary-Enhanced Segment Anything in 3D Gaussian via Gaussian Decomposition](https://mlsanity.com/paper/2401.17857/) (2024)
- [Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mapping](https://mlsanity.com/paper/2111.11348/) (2021)
- [No Pain, Big Gain: Classify Dynamic Point Cloud Sequences with Static Models by Fitting Feature-level Space-time Surfaces](https://mlsanity.com/paper/2203.11113/) (2022)
- [Multi-Space Alignments Towards Universal LiDAR Segmentation](https://mlsanity.com/paper/2405.01538/) (2024)
- [Building3D: An Urban-Scale Dataset and Benchmarks for Learning Roof Structures from Point Clouds](https://mlsanity.com/paper/2307.11914/) (2023)
- [SUM: A Benchmark Dataset of Semantic Urban Meshes](https://mlsanity.com/paper/2103.00355/) (2021)
- [Clustering based Point Cloud Representation Learning for 3D Analysis](https://mlsanity.com/paper/2307.14605/) (2023)
- [CityGen: Infinite and Controllable City Layout Generation](https://mlsanity.com/paper/2312.01508/) (2023)
- [LCPFormer: Towards Effective 3D Point Cloud Analysis via Local Context Propagation in Transformers](https://mlsanity.com/paper/2210.12755/) (2022)
- [A Survey of Label-Efficient Deep Learning for 3D Point Clouds](https://mlsanity.com/paper/2305.19812/) (2023)
- [Visual Foundation Models Boost Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2403.10001/) (2024)
- [LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernels](https://mlsanity.com/paper/2403.15173/) (2024)
- [An Empirical Study of Training State-of-the-Art LiDAR Segmentation Models](https://mlsanity.com/paper/2405.14870/) (2024)
- [LargeAD: Large-Scale Cross-Sensor Data Pretraining for Autonomous Driving](https://mlsanity.com/paper/2501.04005/) (2025)
- [Box2Seg: Learning Semantics of 3D Point Clouds with Box-Level Supervision](https://mlsanity.com/paper/2201.02963/) (2022)
- [ECLAIR: A High-Fidelity Aerial LiDAR Dataset for Semantic Segmentation](https://mlsanity.com/paper/2404.10699/) (2024)

## Abstract (arXiv)

An essential prerequisite for unleashing the potential of supervised deep learning algorithms in the area of 3D scene understanding is the availability of large-scale and richly annotated datasets. However, publicly available datasets are either in relative small spatial scales or have limited semantic annotations due to the expensive cost of data acquisition and data annotation, which severely limits the development of fine-grained semantic understanding in the context of 3D point clouds. In this paper, we present an urban-scale photogrammetric point cloud dataset with nearly three billion richly annotated points, which is three times the number of labeled points than the existing largest photogrammetric point cloud dataset. Our dataset consists of large areas from three UK cities, covering about 7.6 km^2 of the city landscape. In the dataset, each 3D point is labeled as one of 13 semantic classes. We extensively evaluate the performance of state-of-the-art algorithms on our dataset and provide a comprehensive analysis of the results. In particular, we identify several key challenges towards urban-scale point cloud understanding. The dataset is available at https://github.com/QingyongHu/SensatUrban.