# 3D Building Reconstruction from Monocular Remote Sensing Images with Multi-level Supervisions

arXiv: https://arxiv.org/abs/2404.04823 · HTML: https://mlsanity.com/paper/2404.04823/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 48 references that are in the map.
- First posted on arXiv 7 Apr 2024.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [Hybrid Task Cascade for Instance Segmentation](https://mlsanity.com/paper/1901.07518/) (2019)
- [DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images](https://mlsanity.com/paper/1805.06561/) (2018)
- [High-Resolution Representations for Labeling Pixels and Regions](https://mlsanity.com/paper/1904.04514/) (2019)
- [Topological Map Extraction from Overhead Images](https://mlsanity.com/paper/1812.01497/) (2018)
- [Continental-Scale Building Detection from High Resolution Satellite Imagery](https://mlsanity.com/paper/2107.12283/) (2021)
- [Structure-Aware Residual Pyramid Network for Monocular Depth Estimation](https://mlsanity.com/paper/1907.06023/) (2019)
- [PolyWorld: Polygonal Building Extraction with Graph Neural Networks in Satellite Images](https://mlsanity.com/paper/2111.15491/) (2021)
- [Multiview-Consistent Semi-Supervised Learning for 3D Human Pose Estimation](https://mlsanity.com/paper/1908.05293/) (2019)
- [SpaceNet MVOI: a Multi-View Overhead Imagery Dataset](https://mlsanity.com/paper/1903.12239/) (2019)
- [Machine-learned Regularization and Polygonization of Building Segmentation Masks](https://mlsanity.com/paper/2007.12587/) (2020)
- [OmniCity: Omnipotent City Understanding with Multi-level and Multi-view Images](https://mlsanity.com/paper/2208.00928/) (2022)
- [Robust Model-based Face Reconstruction through Weakly-Supervised Outlier Segmentation](https://mlsanity.com/paper/2106.09614/) (2021)
- [Elevation Estimation-Driven Building 3D Reconstruction from Single-View Remote Sensing Imagery](https://mlsanity.com/paper/2301.04581/) (2023)
- [Learning Geocentric Object Pose in Oblique Monocular Images](https://mlsanity.com/paper/2007.00729/) (2020)
- [Weakly supervised 3D Reconstruction with Adversarial Constraint](https://mlsanity.com/paper/1705.10904/) (2017)
- [Learning to Extract Building Footprints from Off-Nadir Aerial Images](https://mlsanity.com/paper/2204.13637/) (2022)
- [Semi-Supervised Adversarial Monocular Depth Estimation](https://mlsanity.com/paper/1908.02126/) (2019)
- [Single View Geocentric Pose in the Wild](https://mlsanity.com/paper/2105.08229/) (2021)
- [THE Benchmark: Transferable Representation Learning for Monocular Height Estimation](https://mlsanity.com/paper/2112.14985/) (2021)
- [Hand Pose Estimation through Semi-Supervised and Weakly-Supervised Learning](https://mlsanity.com/paper/1511.06728/) (2015)

## Built on by (most cited first, 13 of 13)

- [Leveraging BEV Paradigm for Ground-to-Aerial Image Synthesis](https://mlsanity.com/paper/2408.01812/) (2024)
- [SynRS3D: A Synthetic Dataset for Global 3D Semantic Understanding from Monocular Remote Sensing Imagery](https://mlsanity.com/paper/2406.18151/) (2024)
- [Enhancing Monocular Height Estimation via Sparse LiDAR-Guided Correction](https://mlsanity.com/paper/2505.06905/) (2025)
- [PolyFootNet: Extracting Polygonal Building Footprints in Off-Nadir Remote Sensing Images](https://mlsanity.com/paper/2408.08645/) (2024)
- [EO-VGGT: Orbital Ray-Conditioned 3D Foundation Models for Satellite Multi-View Reconstruction](https://mlsanity.com/paper/2607.00417/) (2026)
- [ObliCity: A Benchmark and Baseline for Roof-to-Ground Projection Displacement Correction](https://mlsanity.com/paper/2607.25210/) (2026)
- [SatSAM2: Motion-Constrained Video Object Tracking in Satellite Imagery using Promptable SAM2 and Kalman Priors](https://mlsanity.com/paper/2511.18264/) (2025)
- [SAM 3D for 3D Object Reconstruction from Remote Sensing Images](https://mlsanity.com/paper/2512.22452/) (2025)
- [Delaunay Canopy: Building Wireframe Reconstruction from Airborne LiDAR Point Clouds via Delaunay Graph](https://mlsanity.com/paper/2604.02497/) (2026)
- [SiZeUp: Fast 3D Proxy from Aerial Images via Depth Ordinal Loss](https://mlsanity.com/paper/2608.22821/) (2026)
- [Earth-Agent-Pro: Towards Real-World Full-Chain Earth Observation with Agents](https://mlsanity.com/paper/2609.12533/) (2026)
- [Robust 3D Reconstruction from Multi-View Optical Satellite Imagery via Reliability-Aware Height-Evidence Fusion in Gaussian Splatting](https://mlsanity.com/paper/2609.16772/) (2026)
- [LoDEOT: Low-Dimensional and Efficient Offset Tokens for Building Footprint Extraction from Off-Nadir Imagery](https://mlsanity.com/paper/2610.05899/) (2026)

## Abstract (arXiv)

3D building reconstruction from monocular remote sensing images is an important and challenging research problem that has received increasing attention in recent years, owing to its low cost of data acquisition and availability for large-scale applications. However, existing methods rely on expensive 3D-annotated samples for fully-supervised training, restricting their application to large-scale cross-city scenarios. In this work, we propose MLS-BRN, a multi-level supervised building reconstruction network that can flexibly utilize training samples with different annotation levels to achieve better reconstruction results in an end-to-end manner. To alleviate the demand on full 3D supervision, we design two new modules, Pseudo Building Bbox Calculator and Roof-Offset guided Footprint Extractor, as well as new tasks and training strategies for different types of samples. Experimental results on several public and new datasets demonstrate that our proposed MLS-BRN achieves competitive performance using much fewer 3D-annotated samples, and significantly improves the footprint extraction and 3D reconstruction performance compared with current state-of-the-art. The code and datasets of this work will be released at https://github.com/opendatalab/MLS-BRN.git.