# ViTs for SITS: Vision Transformers for Satellite Image Time Series

arXiv: https://arxiv.org/abs/2301.04944 · HTML: https://mlsanity.com/paper/2301.04944/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 76 references that are in the map.
- First posted on arXiv 12 Jan 2023.

## Builds on (most cited first, 30 of 43)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [Image Transformer](https://mlsanity.com/paper/1802.05751/) (2018)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [End-to-End Video Instance Segmentation with Transformers](https://mlsanity.com/paper/2011.14503/) (2020)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation](https://mlsanity.com/paper/2003.07853/) (2020)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Video Transformer Network](https://mlsanity.com/paper/2102.00719/) (2021)
- [OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks](https://mlsanity.com/paper/1312.6229/) (2013)
- [Deep learning in agriculture: A survey](https://mlsanity.com/paper/1807.11809/) (2018)
- [Panoptic Segmentation of Satellite Image Time Series with Convolutional Temporal Attention Networks](https://mlsanity.com/paper/2107.07933/) (2021)

## Built on by (most cited first, 30 of 56)

- [SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery](https://mlsanity.com/paper/2312.10115/) (2023)
- [Lightweight, Pre-trained Transformers for Remote Sensing Timeseries](https://mlsanity.com/paper/2304.14065/) (2023)
- [AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities](https://mlsanity.com/paper/2412.14123/) (2024)
- [OmniSat: Self-Supervised Modality Fusion for Earth Observation](https://mlsanity.com/paper/2404.08351/) (2024)
- [Channel Vision Transformers: An Image Is Worth 1 x 16 x 16 Words](https://mlsanity.com/paper/2309.16108/) (2023)
- [Harnessing Vision Models for Time Series Analysis: A Survey](https://mlsanity.com/paper/2502.08869/) (2025)
- [OpenForest: A data catalogue for machine learning in forest monitoring](https://mlsanity.com/paper/2311.00277/) (2023)
- [Deep Learning for Satellite Image Time Series Analysis: A Review](https://mlsanity.com/paper/2404.03936/) (2024)
- [FUSU: A Multi-temporal-source Land Use Change Segmentation Dataset for Fine-grained Urban Semantic Understanding](https://mlsanity.com/paper/2405.19055/) (2024)
- [Enhancing Feature Diversity Boosts Channel-Adaptive Vision Transformers](https://mlsanity.com/paper/2405.16419/) (2024)
- [Towards Satellite Image Road Graph Extraction: A Global-Scale Dataset and A Novel Method](https://mlsanity.com/paper/2411.16733/) (2024)
- [Capturing Temporal Dynamics in Large-Scale Canopy Tree Height Estimation](https://mlsanity.com/paper/2501.19328/) (2025)
- [Pixel-wise Agricultural Image Time Series Classification: Comparisons and a Deformable Prototype-based Approach](https://mlsanity.com/paper/2303.12533/) (2023)
- [U-TILISE: A Sequence-to-sequence Model for Cloud Removal in Optical Satellite Time Series](https://mlsanity.com/paper/2305.13277/) (2023)
- [PostRainBench: A comprehensive benchmark and a new model for precipitation forecasting](https://mlsanity.com/paper/2310.02676/) (2023)
- [Revisiting the Encoding of Satellite Image Time Series](https://mlsanity.com/paper/2305.02086/) (2023)
- [Satellite Image Time Series Semantic Change Detection: Novel Architecture and Analysis of Domain Shift](https://mlsanity.com/paper/2407.07616/) (2024)
- [Video Prediction Transformers without Recurrence or Convolution](https://mlsanity.com/paper/2410.04733/) (2024)
- [Locally Adaptive Neural 3D Morphable Models](https://mlsanity.com/paper/2401.02937/) (2024)
- [Continuous Urban Change Detection from Satellite Image Time Series with Temporal Feature Refinement and Multi-Task Integration](https://mlsanity.com/paper/2406.17458/) (2024)
- [TiMo: Spatiotemporal Foundation Model for Satellite Image Time Series](https://mlsanity.com/paper/2505.08723/) (2025)
- [SITSMamba for Crop Classification based on Satellite Image Time Series](https://mlsanity.com/paper/2409.09673/) (2024)
- [Exact: Exploring Space-Time Perceptive Clues for Weakly Supervised Satellite Image Time Series Semantic Segmentation](https://mlsanity.com/paper/2412.03968/) (2024)
- [Empirical Study of PEFT techniques for Winter Wheat Segmentation](https://mlsanity.com/paper/2310.01825/) (2023)
- [Unsupervised Time-Series Signal Analysis with Autoencoders and Vision Transformers: A Review of Architectures and Applications](https://mlsanity.com/paper/2504.16972/) (2025)
- [Not Every Tree Is a Forest: Benchmarking Forest Types from Satellite Remote Sensing](https://mlsanity.com/paper/2505.01805/) (2025)
- [SwissCrop25: A National Multi-Year Benchmark for Operational Crop Mapping](https://mlsanity.com/paper/2608.09497/) (2026)
- [Detecting Looted Archaeological Sites from Satellite Image Time Series](https://mlsanity.com/paper/2409.09432/) (2024)
- [MAESTRO: Masked AutoEncoders for Multimodal, Multitemporal, and Multispectral Earth Observation Data](https://mlsanity.com/paper/2508.10894/) (2025)
- [UniTS: Unified Spatio-Temporal Generative Model for Remote Sensing](https://mlsanity.com/paper/2512.04461/) (2025)

## Abstract (arXiv)

In this paper we introduce the Temporo-Spatial Vision Transformer (TSViT), a fully-attentional model for general Satellite Image Time Series (SITS) processing based on the Vision Transformer (ViT). TSViT splits a SITS record into non-overlapping patches in space and time which are tokenized and subsequently processed by a factorized temporo-spatial encoder. We argue, that in contrast to natural images, a temporal-then-spatial factorization is more intuitive for SITS processing and present experimental evidence for this claim. Additionally, we enhance the model's discriminative power by introducing two novel mechanisms for acquisition-time-specific temporal positional encodings and multiple learnable class tokens. The effect of all novel design choices is evaluated through an extensive ablation study. Our proposed architecture achieves state-of-the-art performance, surpassing previous approaches by a significant margin in three publicly available SITS semantic segmentation and classification datasets. All model, training and evaluation codes are made publicly available to facilitate further research.