# A Multi-Task Deep Learning Framework for Building Footprint Segmentation

arXiv: https://arxiv.org/abs/2104.09375 · HTML: https://mlsanity.com/paper/2104.09375/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 15 references that are in the map.
- First posted on arXiv 19 Apr 2021.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Deep learning in remote sensing: a review](https://mlsanity.com/paper/1710.03959/) (2017)
- [One Model To Learn Them All](https://mlsanity.com/paper/1706.05137/) (2017)
- [SpaceNet 6: Multi-Sensor All Weather Mapping Dataset](https://mlsanity.com/paper/2004.06500/) (2020)
- [Multi-Task Learning for Segmentation of Building Footprints with Deep Neural Networks](https://mlsanity.com/paper/1709.05932/) (2017)
- [A Generalized Multi-Task Learning Approach to Stereo DSM Filtering in Urban Areas](https://mlsanity.com/paper/2004.02493/) (2020)
- [Multi-task Learning for Human Settlement Extent Regression and Local Climate Zone Classification](https://mlsanity.com/paper/2011.11452/) (2020)

## Built on by (most cited first, 1 of 1)

- [Seabed-Net: A multi-task network for joint bathymetry estimation and seabed classification from remote sensing imagery in shallow waters](https://mlsanity.com/paper/2510.19329/) (2025)

## Abstract (arXiv)

The task of building footprint segmentation has been well-studied in the context of remote sensing (RS) as it provides valuable information in many aspects, however, difficulties brought by the nature of RS images such as variations in the spatial arrangements and in-consistent constructional patterns require studying further, since it often causes poorly classified segmentation maps. We address this need by designing a joint optimization scheme for the task of building footprint delineation and introducing two auxiliary tasks; image reconstruction and building footprint boundary segmentation with the intent to reveal the common underlying structure to advance the classification accuracy of a single task model under the favor of auxiliary tasks. In particular, we propose a deep multi-task learning (MTL) based unified fully convolutional framework which operates in an end-to-end manner by making use of joint loss function with learnable loss weights considering the homoscedastic uncertainty of each task loss. Experimental results conducted on the SpaceNet6 dataset demonstrate the potential of the proposed MTL framework as it improves the classification accuracy considerably compared to single-task and lesser compounded tasks.