# Exploiting Semantic Scene Reconstruction for Estimating Building Envelope Characteristics

arXiv: https://arxiv.org/abs/2410.22383 · HTML: https://mlsanity.com/paper/2410.22383/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 68 references that are in the map.
- First posted on arXiv 29 Oct 2024.

## Builds on (most cited first, 15 of 15)

- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Nerfstudio: A Modular Framework for Neural Radiance Field Development](https://mlsanity.com/paper/2302.04264/) (2023)
- [MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstruction](https://mlsanity.com/paper/2206.00665/) (2022)
- [Neuralangelo: High-Fidelity Neural Surface Reconstruction](https://mlsanity.com/paper/2306.03092/) (2023)
- [In-Place Scene Labelling and Understanding with Implicit Scene Representation](https://mlsanity.com/paper/2103.15875/) (2021)
- [3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions](https://mlsanity.com/paper/1603.08182/) (2016)
- [Neural RGB-D Surface Reconstruction](https://mlsanity.com/paper/2104.04532/) (2021)
- [MeshDiffusion: Score-based Generative 3D Mesh Modeling](https://mlsanity.com/paper/2303.08133/) (2023)
- [MAP-Net: Multi Attending Path Neural Network for Building Footprint Extraction from Remote Sensed Imagery](https://mlsanity.com/paper/1910.12060/) (2019)
- [Leveraging Thermal Modality to Enhance Reconstruction in Low-Light Conditions](https://mlsanity.com/paper/2403.14053/) (2024)
- [Synthetic 3D Data Generation Pipeline for Geometric Deep Learning in Architecture](https://mlsanity.com/paper/2104.12564/) (2021)
- [A hybrid-model approach for reducing the performance gap in building energy forecasting](https://mlsanity.com/paper/2206.00460/) (2022)
- [Window to Wall Ratio Detection using SegFormer](https://mlsanity.com/paper/2406.02706/) (2024)

## Built on by (most cited first, 3 of 3)

- [Thermoxels: a voxel-based method to generate simulation-ready 3D thermal models](https://mlsanity.com/paper/2504.04448/) (2025)
- [Multi-view Normal and Distance Guidance Gaussian Splatting for Surface Reconstruction](https://mlsanity.com/paper/2508.07701/) (2025)
- [Unpaired RGB-Thermal Gaussian-Splatting Using Visual Geometric Transformers](https://mlsanity.com/paper/2606.05491/) (2026)

## Abstract (arXiv)

Achieving the EU's climate neutrality goal requires retrofitting existing buildings to reduce energy use and emissions. A critical step in this process is the precise assessment of geometric building envelope characteristics to inform retrofitting decisions. Previous methods for estimating building characteristics, such as window-to-wall ratio, building footprint area, and the location of architectural elements, have primarily relied on applying deep-learning-based detection or segmentation techniques on 2D images. However, these approaches tend to focus on planar facade properties, limiting their accuracy and comprehensiveness when analyzing complete building envelopes in 3D. While neural scene representations have shown exceptional performance in indoor scene reconstruction, they remain under-explored for external building envelope analysis. This work addresses this gap by leveraging cutting-edge neural surface reconstruction techniques based on signed distance function (SDF) representations for 3D building analysis. We propose BuildNet3D, a novel framework to estimate geometric building characteristics from 2D image inputs. By integrating SDF-based representation with semantic modality, BuildNet3D recovers fine-grained 3D geometry and semantics of building envelopes, which are then used to automatically extract building characteristics. Our framework is evaluated on a range of complex building structures, demonstrating high accuracy and generalizability in estimating window-to-wall ratio and building footprint. The results underscore the effectiveness of BuildNet3D for practical applications in building analysis and retrofitting.