# Multimodal Remote Sensing Benchmark Datasets for Land Cover Classification with A Shared and Specific Feature Learning Model

arXiv: https://arxiv.org/abs/2105.10196 · HTML: https://mlsanity.com/paper/2105.10196/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 58 references that are in the map.
- First posted on arXiv 21 May 2021.

## Builds on (most cited first, 15 of 15)

- [Deep learning in remote sensing: a review](https://mlsanity.com/paper/1710.03959/) (2017)
- [More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification](https://mlsanity.com/paper/2008.05457/) (2020)
- [Graph Convolutional Networks for Hyperspectral Image Classification](https://mlsanity.com/paper/2008.02457/) (2020)
- [An Augmented Linear Mixing Model to Address Spectral Variability for Hyperspectral Unmixing](https://mlsanity.com/paper/1810.12000/) (2018)
- [Feature Extraction for Hyperspectral Imagery: The Evolution from Shallow to Deep (Overview and Toolbox)](https://mlsanity.com/paper/2003.02822/) (2020)
- [Classification of Hyperspectral and LiDAR Data Using Coupled CNNs](https://mlsanity.com/paper/2002.01144/) (2020)
- [Hyperspectral and Multispectral Image Fusion based on a Sparse Representation](https://mlsanity.com/paper/1409.5729/) (2014)
- [Learnable Manifold Alignment (LeMA) : A Semi-supervised Cross-modality Learning Framework for Land Cover and Land Use Classification](https://mlsanity.com/paper/1901.02838/) (2019)
- [CoSpace: Common Subspace Learning from Hyperspectral-Multispectral Correspondences](https://mlsanity.com/paper/1812.11501/) (2018)
- [Kernel Manifold Alignment](https://mlsanity.com/paper/1504.02338/) (2015)
- [Parallel Multi-Block ADMM with o(1/k) Convergence](https://mlsanity.com/paper/1312.3040/) (2013)
- [Learning Shared Cross-modality Representation Using Multispectral-LiDAR and Hyperspectral Data](https://mlsanity.com/paper/1912.08837/) (2019)
- [Convergence of multi-block Bregman ADMM for nonconvex composite problems](https://mlsanity.com/paper/1505.03063/) (2015)
- [MIMA: MAPPER-Induced Manifold Alignment for Semi-Supervised Fusion of Optical Image and Polarimetric SAR Data](https://mlsanity.com/paper/1906.05512/) (2019)
- [Semisupervised Manifold Alignment of Multimodal Remote Sensing Images](https://mlsanity.com/paper/2104.07803/) (2021)

## Built on by (most cited first, 30 of 43)

- [Deep Multimodal Learning with Missing Modality: A Survey](https://mlsanity.com/paper/2409.07825/) (2024)
- [Deep Learning in Multimodal Remote Sensing Data Fusion: A Comprehensive Review](https://mlsanity.com/paper/2205.01380/) (2022)
- [Multimodal Fusion Transformer for Remote Sensing Image Classification](https://mlsanity.com/paper/2203.16952/) (2022)
- [Self-Supervised Multimodal Learning: A Survey](https://mlsanity.com/paper/2304.01008/) (2023)
- [MMANet: Margin-aware Distillation and Modality-aware Regularization for Incomplete Multimodal Learning](https://mlsanity.com/paper/2304.08028/) (2023)
- [SpectralMamba: Efficient Mamba for Hyperspectral Image Classification](https://mlsanity.com/paper/2404.08489/) (2024)
- [SS-MAE: Spatial-Spectral Masked Auto-Encoder for Multi-Source Remote Sensing Image Classification](https://mlsanity.com/paper/2311.04442/) (2023)
- [A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application](https://mlsanity.com/paper/2310.14277/) (2023)
- [A Comprehensive Survey for Hyperspectral Image Classification: The Evolution from Conventional to Transformers and Mamba Models](https://mlsanity.com/paper/2404.14955/) (2024)
- [Robust Multimodal Learning via Representation Decoupling](https://mlsanity.com/paper/2407.04458/) (2024)
- [There Are No Data Like More Data- Datasets for Deep Learning in Earth Observation](https://mlsanity.com/paper/2310.19231/) (2023)
- [Boosting Multimodal Learning via Disentangled Gradient Learning](https://mlsanity.com/paper/2507.10213/) (2025)
- [A Unified Framework with Multimodal Fine-tuning for Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2410.11160/) (2024)
- [Benchmarking and scaling of deep learning models for land cover image classification](https://mlsanity.com/paper/2111.09451/) (2021)
- [Unbiased Dynamic Multimodal Fusion](https://mlsanity.com/paper/2603.19681/) (2026)
- [CromSS: Cross-modal pre-training with noisy labels for remote sensing image segmentation](https://mlsanity.com/paper/2405.01217/) (2024)
- [Decoupled Audio-Visual Dataset Distillation](https://mlsanity.com/paper/2511.17890/) (2025)
- [Attention Mechanism Meets with Hybrid Dense Network for Hyperspectral Image Classification](https://mlsanity.com/paper/2201.01001/) (2022)
- [Spatial Gated Multi-Layer Perceptron for Land Use and Land Cover Mapping](https://mlsanity.com/paper/2308.05235/) (2023)
- [MetaSegNet: Metadata-collaborative Vision-Language Representation Learning for Semantic Segmentation of Remote Sensing Images](https://mlsanity.com/paper/2312.12735/) (2023)
- [AllSpark: A Multimodal Spatio-Temporal General Intelligence Model with Ten Modalities via Language as a Reference Framework](https://mlsanity.com/paper/2401.00546/) (2023)
- [Single-stream CNN with Learnable Architecture for Multi-source Remote Sensing Data](https://mlsanity.com/paper/2109.06094/) (2021)
- [Shared Manifold Learning Using a Triplet Network for Multiple Sensor Translation and Fusion with Missing Data](https://mlsanity.com/paper/2210.17311/) (2022)
- [Common Practices and Taxonomy in Deep Multi-view Fusion for Remote Sensing Applications](https://mlsanity.com/paper/2301.01200/) (2022)
- [GAMUS: A Geometry-aware Multi-modal Semantic Segmentation Benchmark for Remote Sensing Data](https://mlsanity.com/paper/2305.14914/) (2023)
- [Asymmetric Hash Code Learning for Remote Sensing Image Retrieval](https://mlsanity.com/paper/2201.05772/) (2022)
- [AdaFlow: Opportunistic Inference on Asynchronous Mobile Data with Generalized Affinity Control](https://mlsanity.com/paper/2410.24028/) (2024)
- [M3amba: CLIP-driven Mamba Model for Multi-modal Remote Sensing Classification](https://mlsanity.com/paper/2503.06446/) (2025)
- [Towards Robust Optical-SAR Object Detection under Missing Modalities: A Dynamic Quality-Aware Fusion Framework](https://mlsanity.com/paper/2512.22447/) (2025)
- [MOGeo: Beyond One-to-One Cross-View Object Geo-localization](https://mlsanity.com/paper/2603.13843/) (2026)

## Abstract (arXiv)

As remote sensing (RS) data obtained from different sensors become available largely and openly, multimodal data processing and analysis techniques have been garnering increasing interest in the RS and geoscience community. However, due to the gap between different modalities in terms of imaging sensors, resolutions, and contents, embedding their complementary information into a consistent, compact, accurate, and discriminative representation, to a great extent, remains challenging. To this end, we propose a shared and specific feature learning (S2FL) model. S2FL is capable of decomposing multimodal RS data into modality-shared and modality-specific components, enabling the information blending of multi-modalities more effectively, particularly for heterogeneous data sources. Moreover, to better assess multimodal baselines and the newly-proposed S2FL model, three multimodal RS benchmark datasets, i.e., Houston2013 -- hyperspectral and multispectral data, Berlin -- hyperspectral and synthetic aperture radar (SAR) data, Augsburg -- hyperspectral, SAR, and digital surface model (DSM) data, are released and used for land cover classification. Extensive experiments conducted on the three datasets demonstrate the superiority and advancement of our S2FL model in the task of land cover classification in comparison with previously-proposed state-of-the-art baselines. Furthermore, the baseline codes and datasets used in this paper will be made available freely at https://github.com/danfenghong/ISPRS_S2FL.