# Deep Learning-based Scalable Image-to-3D Facade Parser for Generating Thermal 3D Building Models

arXiv: https://arxiv.org/abs/2508.04406 · HTML: https://mlsanity.com/paper/2508.04406/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 85 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Builds on (most cited first, 16 of 16)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [Object Detection with Deep Learning: A Review](https://mlsanity.com/paper/1807.05511/) (2018)
- [Global Structure-from-Motion Revisited](https://mlsanity.com/paper/2407.20219/) (2024)
- [Lidar for Autonomous Driving: The principles, challenges, and trends for automotive lidar and perception systems](https://mlsanity.com/paper/2004.08467/) (2020)
- [A Survey on Instance Segmentation: State of the art](https://mlsanity.com/paper/2007.00047/) (2020)
- [Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2303.11203/) (2023)
- [Efficient 2D and 3D Facade Segmentation using Auto-Context](https://mlsanity.com/paper/1606.06437/) (2016)
- [Estimating city-level travel patterns using street imagery: a case study of using Google Street View in Britain](https://mlsanity.com/paper/1802.02915/) (2018)
- [Structure-Aware Completion of Photogrammetric Meshes in Urban Road Environment](https://mlsanity.com/paper/2011.11210/) (2020)
- [Improving Facade Parsing with Vision Transformers and Line Integration](https://mlsanity.com/paper/2309.15523/) (2023)

## Abstract (arXiv)

Renovating existing buildings is essential for climate impact. Early-phase renovation planning requires simulations based on thermal 3D models at Level of Detail (LoD) 3, which include features like windows. However, scalable and accurate identification of such features remains a challenge. This paper presents the Scalable Image-to-3D Facade Parser (SI3FP), a pipeline that generates LoD3 thermal models by extracting geometries from images using both computer vision and deep learning. Unlike existing methods relying on segmentation and projection, SI3FP directly models geometric primitives in the orthographic image plane, providing a unified interface while reducing perspective distortions. SI3FP supports both sparse (e.g., Google Street View) and dense (e.g., hand-held camera) data sources. Tested on typical Swedish residential buildings, SI3FP achieved approximately 5% error in window-to-wall ratio estimates, demonstrating sufficient accuracy for early-stage renovation analysis. The pipeline facilitates large-scale energy renovation planning and has broader applications in urban development and planning.