# BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrieval

arXiv: https://arxiv.org/abs/2105.07921 · HTML: https://mlsanity.com/paper/2105.07921/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 16 references that are in the map.
- First posted on arXiv 17 May 2021.

## Builds on (most cited first, 8 of 8)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understanding](https://mlsanity.com/paper/1902.06148/) (2019)
- [More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification](https://mlsanity.com/paper/2008.05457/) (2020)
- [Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities](https://mlsanity.com/paper/2005.01094/) (2020)
- [Metric-Learning based Deep Hashing Network for Content Based Retrieval of Remote Sensing Images](https://mlsanity.com/paper/1904.01258/) (2019)

## Built on by (most cited first, 30 of 92)

- [SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery](https://mlsanity.com/paper/2312.10115/) (2023)
- [CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders](https://mlsanity.com/paper/2311.00566/) (2023)
- [GEO-Bench: Toward Foundation Models for Earth Monitoring](https://mlsanity.com/paper/2306.03831/) (2023)
- [Vision-Language Models in Remote Sensing: Current Progress and Future Trends](https://mlsanity.com/paper/2305.05726/) (2023)
- [MMEarth: Exploring Multi-Modal Pretext Tasks For Geospatial Representation Learning](https://mlsanity.com/paper/2405.02771/) (2024)
- [PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Models](https://mlsanity.com/paper/2412.04204/) (2024)
- [OmniSat: Self-Supervised Modality Fusion for Earth Observation](https://mlsanity.com/paper/2404.08351/) (2024)
- [Feature Guided Masked Autoencoder for Self-supervised Learning in Remote Sensing](https://mlsanity.com/paper/2310.18653/) (2023)
- [Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2311.15153/) (2023)
- [SpectralEarth: Training Hyperspectral Foundation Models at Scale](https://mlsanity.com/paper/2408.08447/) (2024)
- [FLAIR: a Country-Scale Land Cover Semantic Segmentation Dataset From Multi-Source Optical Imagery](https://mlsanity.com/paper/2310.13336/) (2023)
- [Self-supervised Vision Transformers for Joint SAR-optical Representation Learning](https://mlsanity.com/paper/2204.05381/) (2022)
- [Decoupling Common and Unique Representations for Multimodal Self-supervised Learning](https://mlsanity.com/paper/2309.05300/) (2023)
- [FoMo: Multi-Modal, Multi-Scale and Multi-Task Remote Sensing Foundation Models for Forest Monitoring](https://mlsanity.com/paper/2312.10114/) (2023)
- [Artificial Intelligence for Literature Reviews: Opportunities and Challenges](https://mlsanity.com/paper/2402.08565/) (2024)
- [Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretraining](https://mlsanity.com/paper/2405.20462/) (2024)
- [Revisiting pre-trained remote sensing model benchmarks: resizing and normalization matters](https://mlsanity.com/paper/2305.13456/) (2023)
- [SkySense V2: A Unified Foundation Model for Multi-modal Remote Sensing](https://mlsanity.com/paper/2507.13812/) (2025)
- [Self-supervised Audiovisual Representation Learning for Remote Sensing Data](https://mlsanity.com/paper/2108.00688/) (2021)
- [Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works](https://mlsanity.com/paper/2211.08129/) (2022)
- [Self-Supervised Learning for Invariant Representations from Multi-Spectral and SAR Images](https://mlsanity.com/paper/2205.02049/) (2022)
- [A Survey on Remote Sensing Foundation Models: From Vision to Multimodality](https://mlsanity.com/paper/2503.22081/) (2025)
- [Current Trends in Deep Learning for Earth Observation: An Open-source Benchmark Arena for Image Classification](https://mlsanity.com/paper/2207.07189/) (2022)
- [EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLM](https://mlsanity.com/paper/2506.01667/) (2025)
- [On the Effects of Different Types of Label Noise in Multi-Label Remote Sensing Image Classification](https://mlsanity.com/paper/2207.13975/) (2022)
- [There Are No Data Like More Data- Datasets for Deep Learning in Earth Observation](https://mlsanity.com/paper/2310.19231/) (2023)
- [OpenForest: A data catalogue for machine learning in forest monitoring](https://mlsanity.com/paper/2311.00277/) (2023)
- [Benchmarking and scaling of deep learning models for land cover image classification](https://mlsanity.com/paper/2111.09451/) (2021)
- [Composed Image Retrieval for Remote Sensing](https://mlsanity.com/paper/2405.15587/) (2024)
- [TerraMesh: A Planetary Mosaic of Multimodal Earth Observation Data](https://mlsanity.com/paper/2504.11172/) (2025)

## Abstract (arXiv)

This paper presents the multi-modal BigEarthNet (BigEarthNet-MM) benchmark archive made up of 590,326 pairs of Sentinel-1 and Sentinel-2 image patches to support the deep learning (DL) studies in multi-modal multi-label remote sensing (RS) image retrieval and classification. Each pair of patches in BigEarthNet-MM is annotated with multi-labels provided by the CORINE Land Cover (CLC) map of 2018 based on its thematically most detailed Level-3 class nomenclature. Our initial research demonstrates that some CLC classes are challenging to be accurately described by only considering (single-date) BigEarthNet-MM images. In this paper, we also introduce an alternative class-nomenclature as an evolution of the original CLC labels to address this problem. This is achieved by interpreting and arranging the CLC Level-3 nomenclature based on the properties of BigEarthNet-MM images in a new nomenclature of 19 classes. In our experiments, we show the potential of BigEarthNet-MM for multi-modal multi-label image retrieval and classification problems by considering several state-of-the-art DL models. We also demonstrate that the DL models trained from scratch on BigEarthNet-MM outperform those pre-trained on ImageNet, especially in relation to some complex classes, including agriculture and other vegetated and natural environments. We make all the data and the DL models publicly available at https://bigearth.net, offering an important resource to support studies on multi-modal image scene classification and retrieval problems in RS.