# Building3D: An Urban-Scale Dataset and Benchmarks for Learning Roof Structures from Point Clouds

arXiv: https://arxiv.org/abs/2307.11914 · HTML: https://mlsanity.com/paper/2307.11914/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 70 references that are in the map.
- First posted on arXiv 21 Jul 2023.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Graph Neural Networks: A Review of Methods and Applications](https://mlsanity.com/paper/1812.08434/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Data-Efficient Image Recognition with Contrastive Predictive Coding](https://mlsanity.com/paper/1905.09272/) (2019)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds](https://mlsanity.com/paper/1911.11236/) (2019)
- [Multi-view Convolutional Neural Networks for 3D Shape Recognition](https://mlsanity.com/paper/1505.00880/) (2015)
- [Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data](https://mlsanity.com/paper/1908.04616/) (2019)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017)
- [Masked Autoencoders for Point Cloud Self-supervised Learning](https://mlsanity.com/paper/2203.06604/) (2022)
- [Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs](https://mlsanity.com/paper/1711.09869/) (2017)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training](https://mlsanity.com/paper/2205.14401/) (2022)

## Built on by (most cited first, 24 of 24)

- [Scene4U: Hierarchical Layered 3D Scene Reconstruction from Single Panoramic Image for Your Immerse Exploration](https://mlsanity.com/paper/2504.00387/) (2025)
- [EarthCrafter: Scalable 3D Earth Generation via Dual-Sparse Latent Diffusion](https://mlsanity.com/paper/2507.16535/) (2025)
- [PBWR: Parametric Building Wireframe Reconstruction from Aerial LiDAR Point Clouds](https://mlsanity.com/paper/2311.12062/) (2023)
- [TUM2TWIN: Introducing the Large-Scale Multimodal Urban Digital Twin Benchmark Dataset](https://mlsanity.com/paper/2505.07396/) (2025)
- [ZAHA: Introducing the Level of Facade Generalization and the Large-Scale Point Cloud Facade Semantic Segmentation Benchmark Dataset](https://mlsanity.com/paper/2411.04865/) (2024)
- [A boundary-aware point clustering approach in Euclidean and embedding spaces for roof plane segmentation](https://mlsanity.com/paper/2309.03722/) (2023)
- [Explaining Human Preferences via Metrics for Structured 3D Reconstruction](https://mlsanity.com/paper/2503.08208/) (2025)
- [SAT-SKYLINES: 3D Building Generation from Satellite Imagery and Coarse Geometric Priors](https://mlsanity.com/paper/2508.18531/) (2025)
- [APC2Mesh: Bridging the gap from occluded building faccades to full 3D models](https://mlsanity.com/paper/2404.02391/) (2024)
- [RoofDiffusion: Constructing Roofs from Severely Corrupted Point Data via Diffusion](https://mlsanity.com/paper/2404.09290/) (2024)
- [InLUT3D: Challenging real indoor dataset for point cloud analysis](https://mlsanity.com/paper/2408.03338/) (2024)
- [The P3 Dataset: Pixels, Points and Polygons for Multimodal Building Vectorization](https://mlsanity.com/paper/2505.15379/) (2025)
- [SPPSFormer: High-quality Superpoint-based Transformer for Roof Plane Instance Segmentation from Point Clouds](https://mlsanity.com/paper/2505.24475/) (2025)
- [BuildingWorld: A Structured 3D Building Dataset for Urban Foundation Models](https://mlsanity.com/paper/2511.06337/) (2025)
- [BuildAnyPoint: 3D Building Structured Abstraction from Diverse Point Clouds](https://mlsanity.com/paper/2602.23645/) (2026)
- [ARCH2S: Dataset, Benchmark and Challenges for Learning Exterior Architectural Structures from Point Clouds](https://mlsanity.com/paper/2406.01337/) (2024)
- [GeoTexBuild: 3D Building Model Generation from Map Footprints](https://mlsanity.com/paper/2504.08419/) (2025)
- [Methods for evaluating the resolution of 3D data derived from satellite images](https://mlsanity.com/paper/2506.11876/) (2025)
- [BuildingBRep-11K: Precise Multi-Storey B-Rep Building Solids with Rich Layout Metadata](https://mlsanity.com/paper/2506.15718/) (2025)
- [RoofSeg: An edge-aware transformer-based network for end-to-end roof plane segmentation](https://mlsanity.com/paper/2508.19003/) (2025)
- [SYNBUILD-3D: A large, multi-modal, and semantically rich synthetic dataset of 3D building models at Level of Detail 4](https://mlsanity.com/paper/2508.21169/) (2025)
- [Delaunay Canopy: Building Wireframe Reconstruction from Airborne LiDAR Point Clouds via Delaunay Graph](https://mlsanity.com/paper/2604.02497/) (2026)
- [Learned Suppression for 3D Keypoint Detection with a Graph-Transformer Backbone](https://mlsanity.com/paper/2605.15088/) (2026)
- [ArchEGraph: A Large-Scale Graph Dataset for Geometry-Topology-Physics Aligned Building Energy Modeling](https://mlsanity.com/paper/2608.06772/) (2026)

## Abstract (arXiv)

Urban modeling from LiDAR point clouds is an important topic in computer vision, computer graphics, photogrammetry and remote sensing. 3D city models have found a wide range of applications in smart cities, autonomous navigation, urban planning and mapping etc. However, existing datasets for 3D modeling mainly focus on common objects such as furniture or cars. Lack of building datasets has become a major obstacle for applying deep learning technology to specific domains such as urban modeling. In this paper, we present a urban-scale dataset consisting of more than 160 thousands buildings along with corresponding point clouds, mesh and wire-frame models, covering 16 cities in Estonia about 998 Km2. We extensively evaluate performance of state-of-the-art algorithms including handcrafted and deep feature based methods. Experimental results indicate that Building3D has challenges of high intra-class variance, data imbalance and large-scale noises. The Building3D is the first and largest urban-scale building modeling benchmark, allowing a comparison of supervised and self-supervised learning methods. We believe that our Building3D will facilitate future research on urban modeling, aerial path planning, mesh simplification, and semantic/part segmentation etc.