# Spatio-Temporal Transformers for Long-Term NDVI Forecasting

arXiv: https://arxiv.org/abs/2602.01799 · HTML: https://mlsanity.com/paper/2602.01799/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 52 references that are in the map.
- First posted on arXiv 2 Feb 2026.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [iTransformer: Inverted Transformers Are Effective for Time Series Forecasting](https://mlsanity.com/paper/2310.06625/) (2023)
- [Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecasting](https://mlsanity.com/paper/1912.09363/) (2019)
- [SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2207.08051/) (2022)
- [Earthformer: Exploring Space-Time Transformers for Earth System Forecasting](https://mlsanity.com/paper/2207.05833/) (2022)
- [Self-supervised Learning in Remote Sensing: A Review](https://mlsanity.com/paper/2206.13188/) (2022)
- [ChangeMamba: Remote Sensing Change Detection With Spatiotemporal State Space Model](https://mlsanity.com/paper/2404.03425/) (2024)
- [Lightweight, Pre-trained Transformers for Remote Sensing Timeseries](https://mlsanity.com/paper/2304.14065/) (2023)
- [Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2403.05419/) (2024)
- [Self-attention for raw optical Satellite Time Series Classification](https://mlsanity.com/paper/1910.10536/) (2019)
- [Foundation Models for Remote Sensing and Earth Observation: A Survey](https://mlsanity.com/paper/2410.16602/) (2024)
- [Deep Learning for Satellite Image Time Series Analysis: A Review](https://mlsanity.com/paper/2404.03936/) (2024)
- [Long time-series NDVI reconstruction in cloud-prone regions via spatio-temporal tensor completion](https://mlsanity.com/paper/2102.02603/) (2021)
- [Vision Mamba in Remote Sensing: A Comprehensive Survey of Techniques, Applications and Outlook](https://mlsanity.com/paper/2505.00630/) (2025)
- [Self-Supervised In-Domain Representation Learning for Remote Sensing Image Scene Classification](https://mlsanity.com/paper/2302.01793/) (2023)
- [EHCTNet: Enhanced Hybrid of CNN and Transformer Network for Remote Sensing Image Change Detection](https://mlsanity.com/paper/2501.01238/) (2025)
- [Estimating optical vegetation indices and biophysical variables for temperate forests with Sentinel-1 SAR data using machine learning techniques: A case study for Czechia](https://mlsanity.com/paper/2311.07537/) (2023)

## Abstract (arXiv)

Long-term satellite image time series (SITS) analysis in heterogeneous landscapes faces significant challenges, particularly in Mediterranean regions where complex spatial patterns, seasonal variations, and multi-decade environmental changes interact across different scales. This paper presents the Spatio-Temporal Transformer for Long Term Forecasting (STT-LTF ), an extended framework that advances beyond purely temporal analysis to integrate spatial context modeling with temporal sequence prediction. STT-LTF processes multi-scale spatial patches alongside temporal sequences (up to 20 years) through a unified transformer architecture, capturing both local neighborhood relationships and regional climate influences. The framework employs comprehensive self-supervised learning with spatial masking, temporal masking, and horizon sampling strategies, enabling robust model training from 40 years of unlabeled Landsat imagery. Unlike autoregressive approaches, STT-LTF directly predicts arbitrary future time points without error accumulation, incorporating spatial patch embeddings, cyclical temporal encoding, and geographic coordinates to learn complex dependencies across heterogeneous Mediterranean ecosystems. Experimental evaluation on Landsat data (1984-2024) demonstrates that STT-LTF achieves a Mean Absolute Error (MAE) of 0.0328 and R^2 of 0.8412 for next-year predictions, outperforming traditional statistical methods, CNN-based approaches, LSTM networks, and standard transformers. The framework's ability to handle irregular temporal sampling and variable prediction horizons makes it particularly suitable for analysis of heterogeneous landscapes experiencing rapid ecological transitions.