# BuildingNet: Learning to Label 3D Buildings

arXiv: https://arxiv.org/abs/2110.04955 · HTML: https://mlsanity.com/paper/2110.04955/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 69 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 30 of 37)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data](https://mlsanity.com/paper/1908.04616/) (2019)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs](https://mlsanity.com/paper/1711.09869/) (2017)
- [Neural Relational Inference for Interacting Systems](https://mlsanity.com/paper/1802.04687/) (2018)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark](https://mlsanity.com/paper/1704.03847/) (2017)
- [Virtual Multi-view Fusion for 3D Semantic Segmentation](https://mlsanity.com/paper/2007.13138/) (2020)
- [Learning shape correspondence with anisotropic convolutional neural networks](https://mlsanity.com/paper/1605.06437/) (2016)
- [InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset](https://mlsanity.com/paper/1809.00716/) (2018)
- [Toronto-3D: A Large-scale Mobile LiDAR Dataset for Semantic Segmentation of Urban Roadways](https://mlsanity.com/paper/2003.08284/) (2020)
- [SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation](https://mlsanity.com/paper/1612.00606/) (2016)
- [Paris-Lille-3D: a large and high-quality ground truth urban point cloud dataset for automatic segmentation and classification](https://mlsanity.com/paper/1712.00032/) (2017)
- [3D Shape Segmentation with Projective Convolutional Networks](https://mlsanity.com/paper/1612.02808/) (2016)
- [SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentation](https://mlsanity.com/paper/1907.11308/) (2019)
- [PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentation](https://mlsanity.com/paper/1903.00709/) (2019)
- [DualConvMesh-Net: Joint Geodesic and Euclidean Convolutions on 3D Meshes](https://mlsanity.com/paper/2004.01002/) (2020)
- [MeshWalker: Deep Mesh Understanding by Random Walks](https://mlsanity.com/paper/2006.05353/) (2020)

## Built on by (most cited first, 30 of 33)

- [Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generation](https://mlsanity.com/paper/2501.12202/) (2025)
- [SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generation](https://mlsanity.com/paper/2212.04493/) (2022)
- [State of the Art on Diffusion Models for Visual Computing](https://mlsanity.com/paper/2310.07204/) (2023)
- [Make-A-Shape: a Ten-Million-scale 3D Shape Model](https://mlsanity.com/paper/2401.11067/) (2024)
- [UrbanBIS: a Large-scale Benchmark for Fine-grained Urban Building Instance Segmentation](https://mlsanity.com/paper/2305.02627/) (2023)
- [Pandora3D: A Comprehensive Framework for High-Quality 3D Shape and Texture Generation](https://mlsanity.com/paper/2502.14247/) (2025)
- [Aladdin: Zero-Shot Hallucination of Stylized 3D Assets from Abstract Scene Descriptions](https://mlsanity.com/paper/2306.06212/) (2023)
- [Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodings](https://mlsanity.com/paper/2411.08017/) (2024)
- [Sat2RealCity: Geometry-Aware and Appearance-Controllable 3D Urban Generation from Satellite Imagery](https://mlsanity.com/paper/2511.11470/) (2025)
- [BuilDiff: 3D Building Shape Generation using Single-Image Conditional Point Cloud Diffusion Models](https://mlsanity.com/paper/2309.00158/) (2023)
- [PSSNet: Planarity-sensible Semantic Segmentation of Large-scale Urban Meshes](https://mlsanity.com/paper/2202.03209/) (2022)
- [Vitruvio: 3D Building Meshes via Single Perspective Sketches](https://mlsanity.com/paper/2210.13634/) (2022)
- [FaccAID: A Transformer Model for Neuro-Symbolic Facade Reconstruction](https://mlsanity.com/paper/2406.01829/) (2024)
- [SAT-SKYLINES: 3D Building Generation from Satellite Imagery and Coarse Geometric Priors](https://mlsanity.com/paper/2508.18531/) (2025)
- [Deep Sketch-Based 3D Modeling: A Survey](https://mlsanity.com/paper/2603.03287/) (2026)
- [RoofDiffusion: Constructing Roofs from Severely Corrupted Point Data via Diffusion](https://mlsanity.com/paper/2404.09290/) (2024)
- [Multi-task 3D building understanding with multi-modal pretraining](https://mlsanity.com/paper/2306.10146/) (2023)
- [Weighted Poisson-disk Resampling on Large-Scale Point Clouds](https://mlsanity.com/paper/2412.09177/) (2024)
- [Flash3D: Super-scaling Point Transformers through Joint Hardware-Geometry Locality](https://mlsanity.com/paper/2412.16481/) (2024)
- [SUM Parts: Benchmarking Part-Level Semantic Segmentation of Urban Meshes](https://mlsanity.com/paper/2503.15300/) (2025)
- [BuildingWorld: A Structured 3D Building Dataset for Urban Foundation Models](https://mlsanity.com/paper/2511.06337/) (2025)
- [An Instance-Centric Panoptic Occupancy Prediction Benchmark for Autonomous Driving](https://mlsanity.com/paper/2603.27238/) (2026)
- [Towards AI-Architecture Liberty: A Comprehensive Survey on Design and Generation of Virtual Architecture by Deep Learning](https://mlsanity.com/paper/2305.00510/) (2023)
- [MASSTAR: A Multi-Modal and Large-Scale Scene Dataset with a Versatile Toolchain for Surface Prediction and Completion](https://mlsanity.com/paper/2403.11681/) (2024)
- [ARCH2S: Dataset, Benchmark and Challenges for Learning Exterior Architectural Structures from Point Clouds](https://mlsanity.com/paper/2406.01337/) (2024)
- [ArchComplete: Autoregressive 3D Architectural Design Generation with Hierarchical Diffusion-Based Upsampling](https://mlsanity.com/paper/2412.17957/) (2024)
- [BuildingBRep-11K: Precise Multi-Storey B-Rep Building Solids with Rich Layout Metadata](https://mlsanity.com/paper/2506.15718/) (2025)
- [SYNBUILD-3D: A large, multi-modal, and semantically rich synthetic dataset of 3D building models at Level of Detail 4](https://mlsanity.com/paper/2508.21169/) (2025)
- [Cryo-SWAN: the Multi-Scale Wavelet-decomposition-inspired Autoencoder Network for molecular density representation of molecular volumes](https://mlsanity.com/paper/2603.03342/) (2026)
- [GardenDesigner: Encoding Aesthetic Principles into Jiangnan Garden Construction via a Chain of Agents](https://mlsanity.com/paper/2604.01777/) (2026)

## Abstract (arXiv)

We introduce BuildingNet: (a) a large-scale dataset of 3D building models whose exteriors are consistently labeled, (b) a graph neural network that labels building meshes by analyzing spatial and structural relations of their geometric primitives. To create our dataset, we used crowdsourcing combined with expert guidance, resulting in 513K annotated mesh primitives, grouped into 292K semantic part components across 2K building models. The dataset covers several building categories, such as houses, churches, skyscrapers, town halls, libraries, and castles. We include a benchmark for evaluating mesh and point cloud labeling. Buildings have more challenging structural complexity compared to objects in existing benchmarks (e.g., ShapeNet, PartNet), thus, we hope that our dataset can nurture the development of algorithms that are able to cope with such large-scale geometric data for both vision and graphics tasks e.g., 3D semantic segmentation, part-based generative models, correspondences, texturing, and analysis of point cloud data acquired from real-world buildings. Finally, we show that our mesh-based graph neural network significantly improves performance over several baselines for labeling 3D meshes.