# OBSR: Open Benchmark for Spatial Representations

arXiv: https://arxiv.org/abs/2510.05879 · HTML: https://mlsanity.com/paper/2510.05879/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 58 references that are in the map.
- First posted on arXiv 7 Oct 2025.

## Builds on (most cited first, 19 of 19)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [TravelPlanner: A Benchmark for Real-World Planning with Language Agents](https://mlsanity.com/paper/2402.01622/) (2024)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [GEO-Bench: Toward Foundation Models for Earth Monitoring](https://mlsanity.com/paper/2306.03831/) (2023)
- [TorchGeo: Deep Learning With Geospatial Data](https://mlsanity.com/paper/2111.08872/) (2021)
- [SustainBench: Benchmarks for Monitoring the Sustainable Development Goals with Machine Learning](https://mlsanity.com/paper/2111.04724/) (2021)
- [Can Large Language Models be Good Path Planners? A Benchmark and Investigation on Spatial-temporal Reasoning](https://mlsanity.com/paper/2310.03249/) (2023)
- [MobTCast: Leveraging Auxiliary Trajectory Forecasting for Human Mobility Prediction](https://mlsanity.com/paper/2110.01401/) (2021)
- [TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning](https://mlsanity.com/paper/2406.15658/) (2024)
- [Hex2vec -- Context-Aware Embedding H3 Hexagons with OpenStreetMap Tags](https://mlsanity.com/paper/2111.00970/) (2021)
- [Open-TransMind: A New Baseline and Benchmark for 1st Foundation Model Challenge of Intelligent Transportation](https://mlsanity.com/paper/2304.06051/) (2023)
- [Vision Paper: Causal Inference for Interpretable and Robust Machine Learning in Mobility Analysis](https://mlsanity.com/paper/2210.10010/) (2022)
- [SRAI: Towards Standardization of Geospatial AI](https://mlsanity.com/paper/2310.13098/) (2023)
- [Transfer Learning Approach to Bicycle-sharing Systems' Station Location Planning using OpenStreetMap Data](https://mlsanity.com/paper/2111.00990/) (2021)
- [Complementary Fusion of Deep Network and Tree Model for ETA Prediction](https://mlsanity.com/paper/2407.01262/) (2024)
- [Predicting Crime Using Spatial Features](https://mlsanity.com/paper/1803.04474/) (2018)

## Built on by (most cited first, 3 of 3)

- [CITYREP: A Unified Benchmark for Urban Representations Across Cities, Tasks, and Modalities](https://mlsanity.com/paper/2605.26036/) (2026)
- [Quantifying geographic domain shift to decouple the geospatial transferability of human mobility flow generation models](https://mlsanity.com/paper/2608.21567/) (2026)
- [CartoVec: General-Purpose Location Embeddings from Open Map Data](https://mlsanity.com/paper/2610.17190/) (2026)

## Abstract (arXiv)

GeoAI is evolving rapidly, fueled by diverse geospatial datasets like traffic patterns, environmental data, and crowdsourced OpenStreetMap (OSM) information. While sophisticated AI models are being developed, existing benchmarks are often concentrated on single tasks and restricted to a single modality. As such, progress in GeoAI is limited by the lack of a standardized, multi-task, modality-agnostic benchmark for their systematic evaluation. This paper introduces a novel benchmark designed to assess the performance, accuracy, and efficiency of geospatial embedders. Our benchmark is modality-agnostic and comprises 7 distinct datasets from diverse cities across three continents, ensuring generalizability and mitigating demographic biases. It allows for the evaluation of GeoAI embedders on various phenomena that exhibit underlying geographic processes. Furthermore, we establish a simple and intuitive task-oriented model baselines, providing a crucial reference point for comparing more complex solutions.