# A Geolocation-Aware Multimodal Approach for Ecological Prediction

arXiv: https://arxiv.org/abs/2601.08750 · HTML: https://mlsanity.com/paper/2601.08750/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 13 Jan 2026.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [BioCLIP: A Vision Foundation Model for the Tree of Life](https://mlsanity.com/paper/2311.18803/) (2023)
- [SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery](https://mlsanity.com/paper/2311.17179/) (2023)
- [Seeing biodiversity: perspectives in machine learning for wildlife conservation](https://mlsanity.com/paper/2110.12951/) (2021)
- [SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing](https://mlsanity.com/paper/2312.12856/) (2023)
- [Presence-Only Geographical Priors for Fine-Grained Image Classification](https://mlsanity.com/paper/1906.05272/) (2019)
- [A high-resolution canopy height model of the Earth](https://mlsanity.com/paper/2204.08322/) (2022)
- [Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cells](https://mlsanity.com/paper/2003.00824/) (2020)
- [CSP: Self-Supervised Contrastive Spatial Pre-Training for Geospatial-Visual Representations](https://mlsanity.com/paper/2305.01118/) (2023)
- [Geographic Location Encoding with Spherical Harmonics and Sinusoidal Representation Networks](https://mlsanity.com/paper/2310.06743/) (2023)
- [Spatial Implicit Neural Representations for Global-Scale Species Mapping](https://mlsanity.com/paper/2306.02564/) (2023)
- [Vision Foundation Models in Remote Sensing: A Survey](https://mlsanity.com/paper/2408.03464/) (2024)
- [BioCLIP 2: Emergent Properties from Scaling Hierarchical Contrastive Learning](https://mlsanity.com/paper/2505.23883/) (2025)
- [Geo-Aware Networks for Fine-Grained Recognition](https://mlsanity.com/paper/1906.01737/) (2019)
- [Kriging Convolutional Networks](https://mlsanity.com/paper/2306.09463/) (2023)
- [WildSAT: Learning Satellite Image Representations from Wildlife Observations](https://mlsanity.com/paper/2412.14428/) (2024)
- [Fine-grained Population Mapping from Coarse Census Counts and Open Geodata](https://mlsanity.com/paper/2211.04039/) (2022)
- [Feedforward Few-shot Species Range Estimation](https://mlsanity.com/paper/2502.14977/) (2025)
- [EcoWikiRS: Learning Ecological Representation of Satellite Images from Weak Supervision with Species Observations and Wikipedia](https://mlsanity.com/paper/2504.19742/) (2025)

## Abstract (arXiv)

While integrating multiple modalities has the potential to improve environmental monitoring, current approaches struggle to combine data sources with heterogeneous formats or contents. A central difficulty arises when combining continuous gridded data (e.g., remote sensing) with sparse and irregular point observations such as species records. Existing geostatistical and deep-learning-based approaches typically operate on a single modality or focus on spatially aligned inputs, and thus cannot seamlessly overcome this difficulty. We propose a Geolocation-Aware MultiModal Approach (GAMMA), a transformer-based fusion approach designed to integrate heterogeneous ecological data using explicit spatial context. Instead of interpolating observations into a common grid, GAMMA first represents all inputs as location-aware embeddings that preserve spatial relationships between samples. GAMMA dynamically selects relevant neighbours across modalities and spatial scales, enabling the model to jointly exploit continuous remote sensing imagery and sparse geolocated observations. We evaluate GAMMA on the task of predicting 103 environmental variables from the SWECO25 data cube across Switzerland. Inputs combine aerial imagery with biodiversity observations from GBIF and textual habitat descriptions from Wikipedia, provided by the EcoWikiRS dataset. Experiments show that multimodal fusion consistently improves prediction performance over single-modality baselines and that explicit spatial context further enhances model accuracy. The flexible architecture of GAMMA also allows to analyse the contribution of each modality through controlled ablation experiments. These results demonstrate the potential of location-aware multimodal learning for integrating heterogeneous ecological data and for supporting large-scale environmental mapping tasks and biodiversity monitoring.