# Scan2LoD3: Reconstructing semantic 3D building models at LoD3 using ray casting and Bayesian networks

arXiv: https://arxiv.org/abs/2305.06314 · HTML: https://mlsanity.com/paper/2305.06314/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 63 references that are in the map.
- First posted on arXiv 10 May 2023.

## Builds on (most cited first, 10 of 10)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation](https://mlsanity.com/paper/1611.09326/) (2016)
- [VPC-Net: Completion of 3D Vehicles from MLS Point Clouds](https://mlsanity.com/paper/2008.03404/) (2020)
- [Combining visibility analysis and deep learning for refinement of semantic 3D building models by conflict classification](https://mlsanity.com/paper/2303.05998/) (2023)
- [TUM-FAcCADE: Reviewing and enriching point cloud benchmarks for faccade segmentation](https://mlsanity.com/paper/2304.07140/) (2023)

## Built on by (most cited first, 17 of 17)

- [Texture2LoD3: Enabling LoD3 Building Reconstruction With Panoramic Images](https://mlsanity.com/paper/2504.05249/) (2025)
- [TUM2TWIN: Introducing the Large-Scale Multimodal Urban Digital Twin Benchmark Dataset](https://mlsanity.com/paper/2505.07396/) (2025)
- [MLS2LoD3: Refining low LoDs building models with MLS point clouds to reconstruct semantic LoD3 building models](https://mlsanity.com/paper/2402.06288/) (2024)
- [ZAHA: Introducing the Level of Facade Generalization and the Large-Scale Point Cloud Facade Semantic Segmentation Benchmark Dataset](https://mlsanity.com/paper/2411.04865/) (2024)
- [Analyzing the impact of semantic LoD3 building models on image-based vehicle localization](https://mlsanity.com/paper/2407.21432/) (2024)
- [UltrON: Ultrasound Occupancy Networks](https://mlsanity.com/paper/2509.08991/) (2025)
- [MonoCT: Overcoming Monocular 3D Detection Domain Shift with Consistent Teacher Models](https://mlsanity.com/paper/2503.13743/) (2025)
- [CM2LoD3: Reconstructing LoD3 Building Models Using Semantic Conflict Maps](https://mlsanity.com/paper/2508.15672/) (2025)
- [TrueCity: Real and Simulated Urban Data for Cross-Domain 3D Scene Understanding](https://mlsanity.com/paper/2511.07007/) (2025)
- [RADLER: Radar Object Detection Leveraging Semantic 3D City Models and Self-Supervised Radar-Image Learning](https://mlsanity.com/paper/2504.12167/) (2025)
- [UnderOneFacade: Worldwide Facade Semantic Segmentation Benchmark Dataset](https://mlsanity.com/paper/2607.02018/) (2026)
- [Confidence-Driven Facade Refinement of 3D Building Models Using MLS Point Clouds](https://mlsanity.com/paper/2604.03797/) (2026)
- [GS4City: Hierarchical Semantic Gaussian Splatting via City-Model Priors](https://mlsanity.com/paper/2604.11401/) (2026)
- [SemCityLoc: Aerial 6DoF Localization Using Semantic 3D City Models](https://mlsanity.com/paper/2606.27444/) (2026)
- [KODAMA: Multimodal Digital Twin Reconstruction for Urban RF Propagation Modelling](https://mlsanity.com/paper/2609.07298/) (2026)
- [AstraLOD3: Zero-shot multimodal agentic reconstruction of LOD3 building models](https://mlsanity.com/paper/2609.28061/) (2026)
- [Domain Adaptation of SAM 2.1 for Building-Component Segmentation in Image-Based 3D Reconstruction](https://mlsanity.com/paper/2610.12786/) (2026)

## Abstract (arXiv)

Reconstructing semantic 3D building models at the level of detail (LoD) 3 is a long-standing challenge. Unlike mesh-based models, they require watertight geometry and object-wise semantics at the faccade level. The principal challenge of such demanding semantic 3D reconstruction is reliable faccade-level semantic segmentation of 3D input data. We present a novel method, called Scan2LoD3, that accurately reconstructs semantic LoD3 building models by improving faccade-level semantic 3D segmentation. To this end, we leverage laser physics and 3D building model priors to probabilistically identify model conflicts. These probabilistic physical conflicts propose locations of model openings: Their final semantics and shapes are inferred in a Bayesian network fusing multimodal probabilistic maps of conflicts, 3D point clouds, and 2D images. To fulfill demanding LoD3 requirements, we use the estimated shapes to cut openings in 3D building priors and fit semantic 3D objects from a library of faccade objects. Extensive experiments on the TUM city campus datasets demonstrate the superior performance of the proposed Scan2LoD3 over the state-of-the-art methods in faccade-level detection, semantic segmentation, and LoD3 building model reconstruction. We believe our method can foster the development of probability-driven semantic 3D reconstruction at LoD3 since not only the high-definition reconstruction but also reconstruction confidence becomes pivotal for various applications such as autonomous driving and urban simulations.