# Spatioformer: A Geo-encoded Transformer for Large-Scale Plant Species Richness Prediction

arXiv: https://arxiv.org/abs/2410.19256 · HTML: https://mlsanity.com/paper/2410.19256/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 76 references that are in the map.
- First posted on arXiv 25 Oct 2024.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Rethinking Positional Encoding in Language Pre-training](https://mlsanity.com/paper/2006.15595/) (2020)
- [Geography-Aware Self-Supervised Learning](https://mlsanity.com/paper/2011.09980/) (2020)
- [SpectralFormer: Rethinking Hyperspectral Image Classification with Transformers](https://mlsanity.com/paper/2107.02988/) (2021)
- [Long Short-Term Memory-Networks for Machine Reading](https://mlsanity.com/paper/1601.06733/) (2016)
- [A high-resolution canopy height model of the Earth](https://mlsanity.com/paper/2204.08322/) (2022)
- [Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cells](https://mlsanity.com/paper/2003.00824/) (2020)
- [Geographic Location Encoding with Spherical Harmonics and Sinusoidal Representation Networks](https://mlsanity.com/paper/2310.06743/) (2023)
- [A Review of Location Encoding for GeoAI: Methods and Applications](https://mlsanity.com/paper/2111.04006/) (2021)
- [Sphere2Vec: A General-Purpose Location Representation Learning over a Spherical Surface for Large-Scale Geospatial Predictions](https://mlsanity.com/paper/2306.17624/) (2023)
- [Improving Image Classification with Location Context](https://mlsanity.com/paper/1505.03873/) (2015)
- [Positional Encoder Graph Neural Networks for Geographic Data](https://mlsanity.com/paper/2111.10144/) (2021)
- [Effective Sequential Classifier Training for SVM-based Multitemporal Remote Sensing Image Classification](https://mlsanity.com/paper/1706.04719/) (2017)
- [FactoFormer: Factorized Hyperspectral Transformers with Self-Supervised Pretraining](https://mlsanity.com/paper/2309.09431/) (2023)
- [Plant species richness prediction from DESIS hyperspectral data: A comparison study on feature extraction procedures and regression models](https://mlsanity.com/paper/2301.01918/) (2023)
- [Quantitative Assessment of DESIS Hyperspectral Data for Plant Biodiversity Estimation in Australia](https://mlsanity.com/paper/2207.02482/) (2022)

## Built on by (most cited first, 4 of 4)

- [Decadal analysis of sea surface temperature patterns, climatology, and anomalies in temperate coastal waters with Landsat-8 TIRS observations](https://mlsanity.com/paper/2503.05843/) (2025)
- [Dual-Domain Masked Image Modeling: A Self-Supervised Pretraining Strategy Using Spatial and Frequency Domain Masking for Hyperspectral Data](https://mlsanity.com/paper/2505.03220/) (2025)
- [Space filling positionality and the Spiroformer](https://mlsanity.com/paper/2507.08456/) (2025)
- [Retrieval of Coastal Biogeochemical Parameters From Near-Surface Hyperspectral Remote Sensing Reflectance Using Physics-Aware Meta-Learning](https://mlsanity.com/paper/2605.05623/) (2026)

## Abstract (arXiv)

Earth observation data have shown promise in predicting species richness of vascular plants (α-diversity), but extending this approach to large spatial scales is challenging because geographically distant regions may exhibit different compositions of plant species (β-diversity), resulting in a location-dependent relationship between richness and spectral measurements. In order to handle such geolocation dependency, we propose textitSpatioformer, where a novel geolocation encoder is coupled with the transformer model to encode geolocation context into remote sensing imagery. The Spatioformer model compares favourably to state-of-the-art models in richness predictions on a large-scale ground-truth richness dataset (HAVPlot) that consists of 68,170 in-situ richness samples covering diverse landscapes across Australia. The results demonstrate that geolocational information is advantageous in predicting species richness from satellite observations over large spatial scales. With Spatioformer, plant species richness maps over Australia are compiled from Landsat archive for the years from 2015 to 2023. The richness maps produced in this study reveal the spatiotemporal dynamics of plant species richness in Australia, providing supporting evidence to inform effective planning and policy development for plant diversity conservation. Regions of high richness prediction uncertainties are identified, highlighting the need for future in-situ surveys to be conducted in these areas to enhance the prediction accuracy.