# 3D detection of roof sections from a single satellite image and application to LOD2-building reconstruction

arXiv: https://arxiv.org/abs/2307.05409 · HTML: https://mlsanity.com/paper/2307.05409/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 69 references that are in the map.
- First posted on arXiv 11 Jul 2023.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [Deep learning in remote sensing: a review](https://mlsanity.com/paper/1710.03959/) (2017)
- [PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Image](https://mlsanity.com/paper/1812.04072/) (2018)
- [ParSeNet: A Parametric Surface Fitting Network for 3D Point Clouds](https://mlsanity.com/paper/2003.12181/) (2020)
- [Supervised Fitting of Geometric Primitives to 3D Point Clouds](https://mlsanity.com/paper/1811.08988/) (2018)
- [PlaneNet: Piece-wise Planar Reconstruction from a Single RGB Image](https://mlsanity.com/paper/1804.06278/) (2018)
- [Topological Map Extraction from Overhead Images](https://mlsanity.com/paper/1812.01497/) (2018)
- [Single-Image Piece-wise Planar 3D Reconstruction via Associative Embedding](https://mlsanity.com/paper/1902.09777/) (2019)
- [RegNet: Self-Regulated Network for Image Classification](https://mlsanity.com/paper/2101.00590/) (2021)
- [PolyWorld: Polygonal Building Extraction with Graph Neural Networks in Satellite Images](https://mlsanity.com/paper/2111.15491/) (2021)
- [HPNet: Deep Primitive Segmentation Using Hybrid Representations](https://mlsanity.com/paper/2105.10620/) (2021)
- [PlaneFormers: From Sparse View Planes to 3D Reconstruction](https://mlsanity.com/paper/2208.04307/) (2022)
- [CPFN: Cascaded Primitive Fitting Networks for High-Resolution Point Clouds](https://mlsanity.com/paper/2109.00113/) (2021)
- [Intuitive and Efficient Roof Modeling for Reconstruction and Synthesis](https://mlsanity.com/paper/2109.07683/) (2021)
- [ImpliCity: City Modeling from Satellite Images with Deep Implicit Occupancy Fields](https://mlsanity.com/paper/2201.09968/) (2022)
- [Roof-GAN: Learning to Generate Roof Geometry and Relations for Residential Houses](https://mlsanity.com/paper/2012.09340/) (2020)
- [Automated LoD-2 Model Reconstruction from Very-HighResolution Satellite-derived Digital Surface Model and Orthophoto](https://mlsanity.com/paper/2109.03876/) (2021)
- [Oriented Point Sampling for Plane Detection in Unorganized Point Clouds](https://mlsanity.com/paper/1905.02553/) (2019)

## Built on by (most cited first, 1 of 1)

- [Unsupervised Roofline Extraction from True Orthophotos for LoD2 Building Model Reconstruction](https://mlsanity.com/paper/2310.01067/) (2023)

## Abstract (arXiv)

Reconstructing urban areas in 3D out of satellite raster images has been a long-standing and challenging goal of both academical and industrial research. The rare methods today achieving this objective at a Level Of Details 2 rely on procedural approaches based on geometry, and need stereo images and/or LIDAR data as input. We here propose a method for urban 3D reconstruction named KIBS(textitKeypoints Inference By Segmentation), which comprises two novel features: i) a full deep learning approach for the 3D detection of the roof sections, and ii) only one single (non-orthogonal) satellite raster image as model input. This is achieved in two steps: i) by a Mask R-CNN model performing a 2D segmentation of the buildings' roof sections, and after blending these latter segmented pixels within the RGB satellite raster image, ii) by another identical Mask R-CNN model inferring the heights-to-ground of the roof sections' corners via panoptic segmentation, unto full 3D reconstruction of the buildings and city. We demonstrate the potential of the KIBS method by reconstructing different urban areas in a few minutes, with a Jaccard index for the 2D segmentation of individual roof sections of 88.55% and 75.21% on our two data sets resp., and a height's mean error of such correctly segmented pixels for the 3D reconstruction of 1.60 m and 2.06 m on our two data sets resp., hence within the LOD2 precision range.