# Urban Land Cover Classification with Missing Data Modalities Using Deep Convolutional Neural Networks

arXiv: https://arxiv.org/abs/1709.07383 · HTML: https://mlsanity.com/paper/1709.07383/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Sep 2017.

## Built on by (most cited first, 14 of 14)

- [Deep Multimodal Learning with Missing Modality: A Survey](https://mlsanity.com/paper/2409.07825/) (2024)
- [Transformer Meets Convolution: A Bilateral Awareness Network for Semantic Segmentation of Very Fine Resolution Urban Scene Images](https://mlsanity.com/paper/2106.12413/) (2021)
- [Visible-Infrared Person Re-Identification Using Privileged Intermediate Information](https://mlsanity.com/paper/2209.09348/) (2022)
- [Adaptive Generation of Privileged Intermediate Information for Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2307.03240/) (2023)
- [Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applications](https://mlsanity.com/paper/2301.01200/) (2022)
- [Incomplete Multimodal Learning for Remote Sensing Data Fusion](https://mlsanity.com/paper/2304.11381/) (2023)
- [Multi-modal Co-learning for Earth Observation: Enhancing single-modality models via modality collaboration](https://mlsanity.com/paper/2510.19579/) (2025)
- [Multi-modal land cover mapping of remote sensing images using pyramid attention and gated fusion networks](https://mlsanity.com/paper/2111.03845/) (2021)
- [Filling the Gaps: A Multitask Hybrid Multiscale Generative Framework for Missing Modality in Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2509.11102/) (2025)
- [A Spatially Masked Adaptive Gated Network for multimodal post-flood water extent mapping using SAR and incomplete multispectral data](https://mlsanity.com/paper/2601.00123/) (2025)
- [A2-FPN for Semantic Segmentation of Fine-Resolution Remotely Sensed Images](https://mlsanity.com/paper/2102.07997/) (2021)
- [On What Depends the Robustness of Multi-source Models to Missing Data in Earth Observation?](https://mlsanity.com/paper/2503.19719/) (2025)
- [DIS2: Disentanglement Meets Distillation with Classwise Attention for Robust Remote Sensing Segmentation under Missing Modalities](https://mlsanity.com/paper/2601.13502/) (2026)
- [STARS: Shared-specific Translation and Alignment for missing-modality Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2601.17342/) (2026)

## Abstract (arXiv)

Automatic urban land cover classification is a fundamental problem in remote sensing, e.g. for environmental monitoring. The problem is highly challenging, as classes generally have high inter-class and low intra-class variance. Techniques to improve urban land cover classification performance in remote sensing include fusion of data from different sensors with different data modalities. However, such techniques require all modalities to be available to the classifier in the decision-making process, i.e. at test time, as well as in training. If a data modality is missing at test time, current state-of-the-art approaches have in general no procedure available for exploiting information from these modalities. This represents a waste of potentially useful information. We propose as a remedy a convolutional neural network (CNN) architecture for urban land cover classification which is able to embed all available training modalities in a so-called hallucination network. The network will in effect replace missing data modalities in the test phase, enabling fusion capabilities even when data modalities are missing in testing. We demonstrate the method using two datasets consisting of optical and digital surface model (DSM) images. We simulate missing modalities by assuming that DSM images are missing during testing. Our method outperforms both standard CNNs trained only on optical images as well as an ensemble of two standard CNNs. We further evaluate the potential of our method to handle situations where only some DSM images are missing during testing. Overall, we show that we can clearly exploit training time information of the missing modality during testing.