# Lightweight Temporal Self-Attention for Classifying Satellite Image Time Series

arXiv: https://arxiv.org/abs/2007.00586 · HTML: https://mlsanity.com/paper/2007.00586/

## Key facts

- Cited by 55 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jul 2020.

## Built on by (most cited first, 30 of 55)

- [Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applications](https://mlsanity.com/paper/2412.02732/) (2024)
- [Panoptic Segmentation of Satellite Image Time Series with Convolutional Temporal Attention Networks](https://mlsanity.com/paper/2107.07933/) (2021)
- [AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities](https://mlsanity.com/paper/2412.14123/) (2024)
- [Deep Learning for Time Series Classification and Extrinsic Regression: A Current Survey](https://mlsanity.com/paper/2302.02515/) (2023)
- [PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Models](https://mlsanity.com/paper/2412.04204/) (2024)
- [OmniSat: Self-Supervised Modality Fusion for Earth Observation](https://mlsanity.com/paper/2404.08351/) (2024)
- [Multi-Modal Temporal Attention Models for Crop Mapping from Satellite Time Series](https://mlsanity.com/paper/2112.07558/) (2021)
- [TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis](https://mlsanity.com/paper/2506.20380/) (2025)
- [UnCRtainTS: Uncertainty Quantification for Cloud Removal in Optical Satellite Time Series](https://mlsanity.com/paper/2304.05464/) (2023)
- [TimeMatch: Unsupervised Cross-Region Adaptation by Temporal Shift Estimation](https://mlsanity.com/paper/2111.02682/) (2021)
- [A Survey on Remote Sensing Foundation Models: From Vision to Multimodality](https://mlsanity.com/paper/2503.22081/) (2025)
- [Generalized Classification of Satellite Image Time Series with Thermal Positional Encoding](https://mlsanity.com/paper/2203.09175/) (2022)
- [Deep Learning for Satellite Image Time Series Analysis: A Review](https://mlsanity.com/paper/2404.03936/) (2024)
- [Effective Cloud Removal for Remote Sensing Images by an Improved Mean-Reverting Denoising Model with Elucidated Design Space](https://mlsanity.com/paper/2503.23717/) (2025)
- [GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representations](https://mlsanity.com/paper/2503.16683/) (2025)
- [SMARTIES: Spectrum-Aware Multi-Sensor Auto-Encoder for Remote Sensing Images](https://mlsanity.com/paper/2506.19585/) (2025)
- [TCCT: Tightly-Coupled Convolutional Transformer on Time Series Forecasting](https://mlsanity.com/paper/2108.12784/) (2021)
- [Pixel-wise Agricultural Image Time Series Classification: Comparisons and a Deformable Prototype-based Approach](https://mlsanity.com/paper/2303.12533/) (2023)
- [Adaptive Fusion of Multi-view Remote Sensing data for Optimal Sub-field Crop Yield Prediction](https://mlsanity.com/paper/2401.11844/) (2024)
- [Improved Wildfire Spread Prediction with Time-Series Data and the WSTS+ Benchmark](https://mlsanity.com/paper/2502.12003/) (2025)
- [Location-aware Adaptive Normalization: A Deep Learning Approach For Wildfire Danger Forecasting](https://mlsanity.com/paper/2212.08208/) (2022)
- [FireCastNet: Earth-as-a-Graph for Seasonal Fire Prediction](https://mlsanity.com/paper/2502.01550/) (2025)
- [Revisiting the Encoding of Satellite Image Time Series](https://mlsanity.com/paper/2305.02086/) (2023)
- [Cross-sensor super-resolution of irregularly sampled Sentinel-2 time series](https://mlsanity.com/paper/2404.16409/) (2024)
- [Satellite Image Time Series Semantic Change Detection: Novel Architecture and Analysis of Domain Shift](https://mlsanity.com/paper/2407.07616/) (2024)
- [Paving the way toward foundation models for irregular and unaligned Satellite Image Time Series](https://mlsanity.com/paper/2407.08448/) (2024)
- [Crop Rotation Modeling for Deep Learning-Based Parcel Classification from Satellite Time Series](https://mlsanity.com/paper/2110.08187/) (2021)
- [Continuous Urban Change Detection from Satellite Image Time Series with Temporal Feature Refinement and Multi-Task Integration](https://mlsanity.com/paper/2406.17458/) (2024)
- [RAMEN: Resolution-Adjustable Multimodal Encoder for Earth Observation](https://mlsanity.com/paper/2512.05025/) (2025)
- [Exploring Self-Attention for Crop-type Classification Explainability](https://mlsanity.com/paper/2210.13167/) (2022)

## Abstract (arXiv)

The increasing accessibility and precision of Earth observation satellite data offers considerable opportunities for industrial and state actors alike. This calls however for efficient methods able to process time-series on a global scale. Building on recent work employing multi-headed self-attention mechanisms to classify remote sensing time sequences, we propose a modification of the Temporal Attention Encoder. In our network, the channels of the temporal inputs are distributed among several compact attention heads operating in parallel. Each head extracts highly-specialized temporal features which are in turn concatenated into a single representation. Our approach outperforms other state-of-the-art time series classification algorithms on an open-access satellite image dataset, while using significantly fewer parameters and with a reduced computational complexity.