# A Novel Recurrent Encoder-Decoder Structure for Large-Scale Multi-view Stereo Reconstruction from An Open Aerial Dataset

arXiv: https://arxiv.org/abs/2003.00637 · HTML: https://mlsanity.com/paper/2003.00637/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Mar 2020.

## Built on by (most cited first, 28 of 28)

- [Capturing, Reconstructing, and Simulating: the UrbanScene3D Dataset](https://mlsanity.com/paper/2107.04286/) (2021)
- [S2Looking: A Satellite Side-Looking Dataset for Building Change Detection](https://mlsanity.com/paper/2107.09244/) (2021)
- [EarthNets: Empowering AI in Earth Observation](https://mlsanity.com/paper/2210.04936/) (2022)
- [A Decomposition Model for Stereo Matching](https://mlsanity.com/paper/2104.07516/) (2021)
- [Unsupervised Change Detection Based on Image Reconstruction Loss](https://mlsanity.com/paper/2204.01200/) (2022)
- [Self-Pair: Synthesizing Changes from Single Source for Object Change Detection in Remote Sensing Imagery](https://mlsanity.com/paper/2212.10236/) (2022)
- [RingMo-Aerial: An Aerial Remote Sensing Foundation Model With Affine Transformation Contrastive Learning](https://mlsanity.com/paper/2409.13366/) (2024)
- [Rational Polynomial Camera Model Warping for Deep Learning Based Satellite Multi-View Stereo Matching](https://mlsanity.com/paper/2109.11121/) (2021)
- [ResDepth: A Deep Residual Prior For 3D Reconstruction From High-resolution Satellite Images](https://mlsanity.com/paper/2106.08107/) (2021)
- [Remote Sensing Novel View Synthesis with Implicit Multiplane Representations](https://mlsanity.com/paper/2205.08908/) (2022)
- [Scale Efficient Training for Large Datasets](https://mlsanity.com/paper/2503.13385/) (2025)
- [ORStereo: Occlusion-Aware Recurrent Stereo Matching for 4K-Resolution Images](https://mlsanity.com/paper/2103.07798/) (2021)
- [3D Gaussian Splatting for Large-scale Surface Reconstruction from Aerial Images](https://mlsanity.com/paper/2409.00381/) (2024)
- [Multi-view Gradient Consistency for SVBRDF Estimation of Complex Scenes under Natural Illumination](https://mlsanity.com/paper/2202.13017/) (2022)
- [Rethinking Disparity: A Depth Range Free Multi-View Stereo Based on Disparity](https://mlsanity.com/paper/2211.16905/) (2022)
- [UAVStereo: A Multiple Resolution Dataset for Stereo Matching in UAV Scenarios](https://mlsanity.com/paper/2302.10082/) (2023)
- [SDL-MVS: View Space and Depth Deformable Learning Paradigm for Multi-View Stereo Reconstruction in Remote Sensing](https://mlsanity.com/paper/2405.17140/) (2024)
- [A Global Depth-Range-Free Multi-View Stereo Transformer Network with Pose Embedding](https://mlsanity.com/paper/2411.01893/) (2024)
- [UAVFF3D: A Geometry-Aware Benchmark for Feed-Forward UAV 3D Reconstruction](https://mlsanity.com/paper/2605.17942/) (2026)
- [Mesh Reconstruction from Aerial Images for Outdoor Terrain Mapping Using Joint 2D-3D Learning](https://mlsanity.com/paper/2101.01844/) (2021)
- [EA-3DGS: Efficient and Adaptive 3D Gaussians with Highly Enhanced Quality for outdoor scenes](https://mlsanity.com/paper/2505.10787/) (2025)
- [SplatCo: Structure-View Collaborative Gaussian Splatting for Detail-Preserving Rendering of Large-Scale Unbounded Scenes](https://mlsanity.com/paper/2505.17951/) (2025)
- [Aerial Multi-View Stereo via Adaptive Depth Range Inference and Normal Cues](https://mlsanity.com/paper/2506.05655/) (2025)
- [Enhanced Object Detection in Floor-plan through Super Resolution](https://mlsanity.com/paper/2112.09844/) (2021)
- [TerrainMesh: Metric-Semantic Terrain Reconstruction from Aerial Images Using Joint 2D-3D Learning](https://mlsanity.com/paper/2204.10993/) (2022)
- [Batch Loss Score for Dynamic Data Pruning](https://mlsanity.com/paper/2604.04681/) (2026)
- [DAPM: UAV Monocular Depth Estimation from Any Height, Pitch, Roll and FOV](https://mlsanity.com/paper/2607.21438/) (2026)
- [TopoGS: Topology-Aware Anchor Feature Aggregation for Large-Scale 3D Gaussian Splatting](https://mlsanity.com/paper/2609.27868/) (2026)

## Abstract (arXiv)

A great deal of research has demonstrated recently that multi-view stereo (MVS) matching can be solved with deep learning methods. However, these efforts were focused on close-range objects and only a very few of the deep learning-based methods were specifically designed for large-scale 3D urban reconstruction due to the lack of multi-view aerial image benchmarks. In this paper, we present a synthetic aerial dataset, called the WHU dataset, we created for MVS tasks, which, to our knowledge, is the first large-scale multi-view aerial dataset. It was generated from a highly accurate 3D digital surface model produced from thousands of real aerial images with precise camera parameters. We also introduce in this paper a novel network, called RED-Net, for wide-range depth inference, which we developed from a recurrent encoder-decoder structure to regularize cost maps across depths and a 2D fully convolutional network as framework. RED-Net's low memory requirements and high performance make it suitable for large-scale and highly accurate 3D Earth surface reconstruction. Our experiments confirmed that not only did our method exceed the current state-of-the-art MVS methods by more than 50% mean absolute error (MAE) with less memory and computational cost, but its efficiency as well. It outperformed one of the best commercial software programs based on conventional methods, improving their efficiency 16 times over. Moreover, we proved that our RED-Net model pre-trained on the synthetic WHU dataset can be efficiently transferred to very different multi-view aerial image datasets without any fine-tuning. Dataset are available at http://gpcv.whu.edu.cn/data.