# M3LEO: A Multi-Modal, Multi-Label Earth Observation Dataset Integrating Interferometric SAR and Multispectral Data arXiv: https://arxiv.org/abs/2406.04230 ยท HTML: https://mlsanity.com/paper/2406.04230/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Builds on 34 of its 110 references that are in the map. - First posted on arXiv 6 Jun 2024. ## Builds on (most cited first, 30 of 34) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023) - [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021) - [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017) - [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020) - [Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks](https://mlsanity.com/paper/2208.10442/) (2022) - [InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions](https://mlsanity.com/paper/2211.05778/) (2022) - [BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understanding](https://mlsanity.com/paper/1902.06148/) (2019) - [Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data](https://mlsanity.com/paper/2103.16607/) (2021) - [SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery](https://mlsanity.com/paper/2312.10115/) (2023) - [SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understanding](https://mlsanity.com/paper/2211.15660/) (2022) - [Self-supervised Learning in Remote Sensing: A Review](https://mlsanity.com/paper/2206.13188/) (2022) - [GEO-Bench: Toward Foundation Models for Earth Monitoring](https://mlsanity.com/paper/2306.03831/) (2023) - [SEN12MS -- A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion](https://mlsanity.com/paper/1906.07789/) (2019) - [SSL4EO-L: Datasets and Foundation Models for Landsat Imagery](https://mlsanity.com/paper/2306.09424/) (2023) - [The SEN1-2 Dataset for Deep Learning in SAR-Optical Data Fusion](https://mlsanity.com/paper/1807.01569/) (2018) - [Open High-Resolution Satellite Imagery: The WorldStrat Dataset -- With Application to Super-Resolution](https://mlsanity.com/paper/2207.06418/) (2022) - [Major TOM: Expandable Datasets for Earth Observation](https://mlsanity.com/paper/2402.12095/) (2024) - [Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works](https://mlsanity.com/paper/2211.08129/) (2022) - [Active Fire Detection in Landsat-8 Imagery: a Large-Scale Dataset and a Deep-Learning Study](https://mlsanity.com/paper/2101.03409/) (2021) - [A Sentinel-2 multi-year, multi-country benchmark dataset for crop classification and segmentation with deep learning](https://mlsanity.com/paper/2204.00951/) (2022) - [EarthPT: a time series foundation model for Earth Observation](https://mlsanity.com/paper/2309.07207/) (2023) - [Hephaestus: A large scale multitask dataset towards InSAR understanding](https://mlsanity.com/paper/2204.09435/) (2022) - [Space-based Global Maritime Surveillance. Part I: Satellite Technologies](https://mlsanity.com/paper/2011.11304/) (2020) - [Large Scale Masked Autoencoding for Reducing Label Requirements on SAR Data](https://mlsanity.com/paper/2310.00826/) (2023) - [Improving Landslide Detection on SAR Data through Deep Learning](https://mlsanity.com/paper/2105.00782/) (2021) - [Fewshot learning on global multimodal embeddings for earth observation tasks](https://mlsanity.com/paper/2310.00119/) (2023) ## Built on by (most cited first, 6 of 6) - [REO-VLM: Transforming VLM to Meet Regression Challenges in Earth Observation](https://mlsanity.com/paper/2412.16583/) (2024) - [LiDAR Remote Sensing Meets Weak Supervision: Concepts, Methods, and Perspectives](https://mlsanity.com/paper/2503.18384/) (2025) - [OS-W2S: An Automatic Labeling Engine for Language-Guided Open-Set Aerial Object Detection](https://mlsanity.com/paper/2505.03334/) (2025) - [Probabilistic Hyper-Graphs using Multiple Randomly Masked Autoencoders for Semi-supervised Multi-modal Multi-task Learning](https://mlsanity.com/paper/2510.10068/) (2025) - [Context-Aware Multimodal Representation Learning for Spatio-Temporally Explicit Environmental Modelling](https://mlsanity.com/paper/2511.11706/) (2025) - [Position: AI for Science Should Treat Measurement-to-Dataset Pipelines as Inference Components](https://mlsanity.com/paper/2605.24558/) (2026) ## Abstract (arXiv) Satellite-based remote sensing has revolutionised the way we address global challenges. Huge quantities of Earth Observation (EO) data are generated by satellite sensors daily, but processing these large datasets for use in ML pipelines is technically and computationally challenging. While some preprocessed Earth observation datasets exist, their content is often limited to optical or near-optical wavelength data, which is ineffective at night or in adverse weather conditions. Synthetic Aperture Radar (SAR), an active sensing technique based on microwave length radiation, offers a viable alternative. However, the application of machine learning to SAR has been limited due to a lack of ML-ready data and pipelines, particularly for the full diversity of SAR data, including polarimetry, coherence and interferometry. In this work, we introduce M3LEO, a multi-modal, multi-label Earth observation dataset that includes polarimetric, interferometric, and coherence SAR data derived from Sentinel-1, alongside multispectral Sentinel-2 imagery and auxiliary data describing terrain properties such as land use. M3LEO spans approximately 17M 4x4 km data chips from six diverse geographic regions. The dataset is complemented by a flexible PyTorch Lightning framework configured using Hydra to accommodate its use across diverse ML applications in Earth observation. We provide tools to process any dataset available on popular platforms such as Google Earth Engine for seamless integration with our framework. We show that the distribution shift in self-supervised embeddings is substantial across geographic regions, even when controlling for terrain properties. Data: huggingface.co/M3LEO, Code: github.com/spaceml-org/M3LEO.