# A Generalized Multi-Task Learning Approach to Stereo DSM Filtering in Urban Areas

arXiv: https://arxiv.org/abs/2004.02493 · HTML: https://mlsanity.com/paper/2004.02493/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2020.

## Built on by (most cited first, 5 of 5)

- [Sat-NeRF: Learning Multi-View Satellite Photogrammetry With Transient Objects and Shadow Modeling Using RPC Cameras](https://mlsanity.com/paper/2203.08896/) (2022)
- [ResDepth: A Deep Residual Prior For 3D Reconstruction From High-resolution Satellite Images](https://mlsanity.com/paper/2106.08107/) (2021)
- [Machine-learned 3D Building Vectorization from Satellite Imagery](https://mlsanity.com/paper/2104.06485/) (2021)
- [A Multi-Task Deep Learning Framework for Building Footprint Segmentation](https://mlsanity.com/paper/2104.09375/) (2021)
- [Estimating Residential Solar Potential Using Aerial Data](https://mlsanity.com/paper/2306.13564/) (2023)

## Abstract (arXiv)

City models and height maps of urban areas serve as a valuable data source for numerous applications, such as disaster management or city planning. While this information is not globally available, it can be substituted by digital surface models (DSMs), automatically produced from inexpensive satellite imagery. However, stereo DSMs often suffer from noise and blur. Furthermore, they are heavily distorted by vegetation, which is of lesser relevance for most applications. Such basic models can be filtered by convolutional neural networks (CNNs), trained on labels derived from digital elevation models (DEMs) and 3D city models, in order to obtain a refined DSM. We propose a modular multi-task learning concept that consolidates existing approaches into a generalized framework. Our encoder-decoder models with shared encoders and multiple task-specific decoders leverage roof type classification as a secondary task and multiple objectives including a conditional adversarial term. The contributing single-objective losses are automatically weighted in the final multi-task loss function based on learned uncertainty estimates. We evaluated the performance of specific instances of this family of network architectures. Our method consistently outperforms the state of the art on common data, both quantitatively and qualitatively, and generalizes well to a new dataset of an independent study area.