# Multi-Modal Deep Learning for Multi-Temporal Urban Mapping With a Partly Missing Optical Modality

arXiv: https://arxiv.org/abs/2306.00640 · HTML: https://mlsanity.com/paper/2306.00640/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 11 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 3 of 3)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Multi-Temporal Urban Development SpaceNet Dataset](https://mlsanity.com/paper/2102.04420/) (2021)
- [Overcoming Missing and Incomplete Modalities with Generative Adversarial Networks for Building Footprint Segmentation](https://mlsanity.com/paper/1808.03195/) (2018)

## Built on by (most cited first, 1 of 1)

- [Deep Multimodal Learning with Missing Modality: A Survey](https://mlsanity.com/paper/2409.07825/) (2024)

## Abstract (arXiv)

This paper proposes a novel multi-temporal urban mapping approach using multi-modal satellite data from the Sentinel-1 Synthetic Aperture Radar (SAR) and Sentinel-2 MultiSpectral Instrument (MSI) missions. In particular, it focuses on the problem of a partly missing optical modality due to clouds. The proposed model utilizes two networks to extract features from each modality separately. In addition, a reconstruction network is utilized to approximate the optical features based on the SAR data in case of a missing optical modality. Our experiments on a multi-temporal urban mapping dataset with Sentinel-1 SAR and Sentinel-2 MSI data demonstrate that the proposed method outperforms a multi-modal approach that uses zero values as a replacement for missing optical data, as well as a uni-modal SAR-based approach. Therefore, the proposed method is effective in exploiting multi-modal data, if available, but it also retains its effectiveness in case the optical modality is missing.