# Self-attention for raw optical Satellite Time Series Classification

arXiv: https://arxiv.org/abs/1910.10536 · HTML: https://mlsanity.com/paper/1910.10536/

## Key facts

- Cited by 75 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2019.

## Built on by (most cited first, 30 of 75)

- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2207.08051/) (2022)
- [Panoptic Segmentation of Satellite Image Time Series with Convolutional Temporal Attention Networks](https://mlsanity.com/paper/2107.07933/) (2021)
- [DiffusionSat: A Generative Foundation Model for Satellite Imagery](https://mlsanity.com/paper/2312.03606/) (2023)
- [Lightweight, Pre-trained Transformers for Remote Sensing Timeseries](https://mlsanity.com/paper/2304.14065/) (2023)
- [Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey](https://mlsanity.com/paper/2302.02515/) (2023)
- [OmniSat: Self-Supervised Modality Fusion for Earth Observation](https://mlsanity.com/paper/2404.08351/) (2024)
- [Gated Transformer Networks for Multivariate Time Series Classification](https://mlsanity.com/paper/2103.14438/) (2021)
- [ViTs for SITS: Vision Transformers for Satellite Image Time Series](https://mlsanity.com/paper/2301.04944/) (2023)
- [Multi-Modal Temporal Attention Models for Crop Mapping from Satellite Time Series](https://mlsanity.com/paper/2112.07558/) (2021)
- [Transformers in Time-series Analysis: A Tutorial](https://mlsanity.com/paper/2205.01138/) (2022)
- [Crop mapping from image time series: deep learning with multi-scale label hierarchies](https://mlsanity.com/paper/2102.08820/) (2021)
- [UnCRtainTS: Uncertainty Quantification for Cloud Removal in Optical Satellite Time Series](https://mlsanity.com/paper/2304.05464/) (2023)
- [Artificial intelligence to advance Earth observation: : A review of models, recent trends, and pathways forward](https://mlsanity.com/paper/2305.08413/) (2023)
- [TimeMatch: Unsupervised Cross-Region Adaptation by Temporal Shift Estimation](https://mlsanity.com/paper/2111.02682/) (2021)
- [FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classification](https://mlsanity.com/paper/2302.09818/) (2023)
- [A Sentinel-2 multi-year, multi-country benchmark dataset for crop classification and segmentation with deep learning](https://mlsanity.com/paper/2204.00951/) (2022)
- [UrbanVLP: Multi-Granularity Vision-Language Pretraining for Urban Socioeconomic Indicator Prediction](https://mlsanity.com/paper/2403.16831/) (2024)
- [Domain-Adversarial Training of Self-Attention Based Networks for Land Cover Classification using Multi-temporal Sentinel-2 Satellite Imagery](https://mlsanity.com/paper/2104.00564/) (2021)
- [Generalized Classification of Satellite Image Time Series with Thermal Positional Encoding](https://mlsanity.com/paper/2203.09175/) (2022)
- [EarthPT: a time series foundation model for Earth Observation](https://mlsanity.com/paper/2309.07207/) (2023)
- [Deep Learning for Satellite Image Time Series Analysis: A Review](https://mlsanity.com/paper/2404.03936/) (2024)
- [Country-wide Retrieval of Forest Structure From Optical and SAR Satellite Imagery With Deep Ensembles](https://mlsanity.com/paper/2111.13154/) (2021)
- [Pixel-wise Agricultural Image Time Series Classification: Comparisons and a Deformable Prototype-based Approach](https://mlsanity.com/paper/2303.12533/) (2023)
- [Boosting Crop Classification by Hierarchically Fusing Satellite, Rotational, and Contextual Data](https://mlsanity.com/paper/2305.12011/) (2023)
- [U-TILISE: A Sequence-to-sequence Model for Cloud Removal in Optical Satellite Time Series](https://mlsanity.com/paper/2305.13277/) (2023)
- [Sparse Binary Transformers for Multivariate Time Series Modeling](https://mlsanity.com/paper/2308.04637/) (2023)
- [Adaptive Fusion of Multi-view Remote Sensing data for Optimal Sub-field Crop Yield Prediction](https://mlsanity.com/paper/2401.11844/) (2024)
- [AiTLAS: Artificial Intelligence Toolbox for Earth Observation](https://mlsanity.com/paper/2201.08789/) (2022)
- [Predictive Maintenance for General Aviation Using Convolutional Transformers](https://mlsanity.com/paper/2110.03757/) (2021)

## Abstract (arXiv)

The amount of available Earth observation data has increased dramatically in the recent years. Efficiently making use of the entire body information is a current challenge in remote sensing and demands for light-weight problem-agnostic models that do not require region- or problem-specific expert knowledge. End-to-end trained deep learning models can make use of raw sensory data by learning feature extraction and classification in one step solely from data. Still, many methods proposed in remote sensing research require implicit feature extraction through data preprocessing or explicit design of features. In this work, we compare recent deep learning models on crop type classification on raw and preprocessed Sentinel 2 data. We concentrate on the common neural network architectures for time series, i.e., 1D-convolutions, recurrence, a shallow random forest baseline, and focus on the novel self-attention architecture. Our central findings are that data preprocessing still increased the overall classification performance for all models while the choice of model was less crucial. Self-attention and recurrent neural networks, by their architecture, outperformed convolutional neural networks on raw satellite time series. We explore this by a feature importance analysis based on gradient back-propagation that exploits the differentiable nature of deep learning models. Further, we qualitatively show how self-attention scores focus selectively on few classification-relevant observations.