# ResDepth: A Deep Residual Prior For 3D Reconstruction From High-resolution Satellite Images

arXiv: https://arxiv.org/abs/2106.08107 · HTML: https://mlsanity.com/paper/2106.08107/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 64 references that are in the map.
- First posted on arXiv 15 Jun 2021.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [Cascade Cost Volume for High-Resolution Multi-View Stereo and Stereo Matching](https://mlsanity.com/paper/1912.06378/) (2019)
- [End-to-End Learning of Geometry and Context for Deep Stereo Regression](https://mlsanity.com/paper/1703.04309/) (2017)
- [AANet: Adaptive Aggregation Network for Efficient Stereo Matching](https://mlsanity.com/paper/2004.09548/) (2020)
- [Stereo Matching by Training a Convolutional Neural Network to Compare Image Patches](https://mlsanity.com/paper/1510.05970/) (2015)
- [Hierarchical Deep Stereo Matching on High-resolution Images](https://mlsanity.com/paper/1912.06704/) (2019)
- [DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch](https://mlsanity.com/paper/1909.05845/) (2019)
- [Cascade Residual Learning: A Two-stage Convolutional Neural Network for Stereo Matching](https://mlsanity.com/paper/1708.09204/) (2017)
- [FADNet: A Fast and Accurate Network for Disparity Estimation](https://mlsanity.com/paper/2003.10758/) (2020)
- [A Novel Recurrent Encoder-Decoder Structure for Large-Scale Multi-view Stereo Reconstruction from An Open Aerial Dataset](https://mlsanity.com/paper/2003.00637/) (2020)
- [Practical Deep Stereo (PDS): Toward applications-friendly deep stereo matching](https://mlsanity.com/paper/1806.01677/) (2018)
- [A New Stereo Benchmarking Dataset for Satellite Images](https://mlsanity.com/paper/1907.04404/) (2019)
- [Left-Right Comparative Recurrent Model for Stereo Matching](https://mlsanity.com/paper/1804.00796/) (2018)
- [Detect, Replace, Refine: Deep Structured Prediction For Pixel Wise Labeling](https://mlsanity.com/paper/1612.04770/) (2016)
- [Machine-learned 3D Building Vectorization from Satellite Imagery](https://mlsanity.com/paper/2104.06485/) (2021)
- [Late or Earlier Information Fusion from Depth and Spectral Data? Large-Scale Digital Surface Model Refinement by Hybrid-cGAN](https://mlsanity.com/paper/1904.09935/) (2019)
- [ResDepth: Learned Residual Stereo Reconstruction](https://mlsanity.com/paper/2001.08026/) (2020)
- [A Generalized Multi-Task Learning Approach to Stereo DSM Filtering in Urban Areas](https://mlsanity.com/paper/2004.02493/) (2020)
- [DSM Building Shape Refinement from Combined Remote Sensing Images based on Wnet-cGANs](https://mlsanity.com/paper/1903.03519/) (2019)
- [Learning Dense Stereo Matching for Digital Surface Models from Satellite Imagery](https://mlsanity.com/paper/1811.03535/) (2018)

## Built on by (most cited first, 10 of 10)

- [ImpliCity: City Modeling from Satellite Images with Deep Implicit Occupancy Fields](https://mlsanity.com/paper/2201.09968/) (2022)
- [WHU-Stereo: A Challenging Benchmark for Stereo Matching of High-Resolution Satellite Images](https://mlsanity.com/paper/2206.02342/) (2022)
- [Sat-DN: Implicit Surface Reconstruction from Multi-View Satellite Images with Depth and Normal Supervision](https://mlsanity.com/paper/2502.08352/) (2025)
- [GroundUp: Rapid Sketch-Based 3D City Massing](https://mlsanity.com/paper/2407.12739/) (2024)
- [EO-VGGT: Orbital Ray-Conditioned 3D Foundation Models for Satellite Multi-View Reconstruction](https://mlsanity.com/paper/2607.00417/) (2026)
- [Satellite Sunroof: High-res Digital Surface Models and Roof Segmentation for Global Solar Mapping](https://mlsanity.com/paper/2408.14400/) (2024)
- [T-Graph: Enhancing Sparse-view Camera Pose Estimation by Pairwise Translation Graph](https://mlsanity.com/paper/2505.01207/) (2025)
- [RoughNet: Mapping Arctic Sea Ice Roughness Using Diffusion-Based Super-Resolution of Satellite Imagery](https://mlsanity.com/paper/2607.13371/) (2026)
- [SeasonStereo: Robust Dense Stereo Matching for Multi-Date Satellite Imagery via Generative AI](https://mlsanity.com/paper/2607.27139/) (2026)
- [Rethinking Handwritten Character Recognition](https://mlsanity.com/paper/2609.10572/) (2026)

## Abstract (arXiv)

Modern optical satellite sensors enable high-resolution stereo reconstruction from space. But the challenging imaging conditions when observing the Earth from space push stereo matching to its limits. In practice, the resulting digital surface models (DSMs) are fairly noisy and often do not attain the accuracy needed for high-resolution applications such as 3D city modeling. Arguably, stereo correspondence based on low-level image similarity is insufficient and should be complemented with a-priori knowledge about the expected surface geometry beyond basic local smoothness. To that end, we introduce ResDepth, a convolutional neural network that learns such an expressive geometric prior from example data. ResDepth refines an initial, raw stereo DSM while conditioning the refinement on the images. I.e., it acts as a smart, learned post-processing filter and can seamlessly complement any stereo matching pipeline. In a series of experiments, we find that the proposed method consistently improves stereo DSMs both quantitatively and qualitatively. We show that the prior encoded in the network weights captures meaningful geometric characteristics of urban design, which also generalize across different districts and even from one city to another. Moreover, we demonstrate that, by training on a variety of stereo pairs, ResDepth can acquire a sufficient degree of invariance against variations in imaging conditions and acquisition geometry.