# GPSBench: Do Large Language Models Understand GPS Coordinates?

arXiv: https://arxiv.org/abs/2602.16105 · HTML: https://mlsanity.com/paper/2602.16105/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 18 Feb 2026.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 22 of 22)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [Visual Spatial Reasoning](https://mlsanity.com/paper/2205.00363/) (2022)
- [What's "up" with vision-language models? Investigating their struggle with spatial reasoning](https://mlsanity.com/paper/2310.19785/) (2023)
- [LingoQA: Visual Question Answering for Autonomous Driving](https://mlsanity.com/paper/2312.14115/) (2023)
- [Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models](https://mlsanity.com/paper/2406.14852/) (2024)
- [Large Language Models are Geographically Biased](https://mlsanity.com/paper/2402.02680/) (2024)
- [Evaluating Spatial Understanding of Large Language Models](https://mlsanity.com/paper/2310.14540/) (2023)
- [GeoLLM: Extracting Geospatial Knowledge from Large Language Models](https://mlsanity.com/paper/2310.06213/) (2023)
- [Autonomous GIS: the next-generation AI-powered GIS](https://mlsanity.com/paper/2305.06453/) (2023)
- [SpatialBench: Benchmarking Multimodal Large Language Models for Spatial Cognition](https://mlsanity.com/paper/2511.21471/) (2025)
- [MapEval: A Map-Based Evaluation of Geo-Spatial Reasoning in Foundation Models](https://mlsanity.com/paper/2501.00316/) (2024)
- [GeoBenchX: Benchmarking LLMs in Agent Solving Multistep Geospatial Tasks](https://mlsanity.com/paper/2503.18129/) (2025)
- [GeoGramBench: Benchmarking the Geometric Program Reasoning in Modern LLMs](https://mlsanity.com/paper/2505.17653/) (2025)
- [GeoAnalystBench: A GeoAI benchmark for assessing large language models for spatial analysis workflow and code generation](https://mlsanity.com/paper/2509.05881/) (2025)
- [GeoGLUE: A GeoGraphic Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2305.06545/) (2023)
- [Distortions in Judged Spatial Relations in Large Language Models](https://mlsanity.com/paper/2401.04218/) (2024)

## Built on by (most cited first, 4 of 4)

- [GeoNatureAgent Benchmark: Benchmarking LLM Agents for Environmental Geospatial Analysis Across Frontier and Open-Weight Foundation Models](https://mlsanity.com/paper/2606.12821/) (2026)
- [MultiGlobeQA: A Multilingual and Globally Diverse Benchmark for Geospatial Reasoning](https://mlsanity.com/paper/2608.03882/) (2026)
- [GeoNatureAgent (GNA): A Framework and Benchmark for Pre-Production Evaluation of Tool-Using Agents on Geospatial and Environmental Tasks](https://mlsanity.com/paper/2610.09112/) (2026)
- [How Reliable Are Large Language Models for Soil Organic Carbon Assessment? An Australian Benchmark](https://mlsanity.com/paper/2610.18201/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly deployed in applications that interact with the physical world, such as navigation, robotics, or mapping, making robust geospatial reasoning a critical capability. Despite that, LLMs' ability to reason about GPS coordinates and real-world geography remains underexplored. We introduce GPSBench, a dataset of 57,800 samples across 17 tasks for evaluating geospatial reasoning in LLMs, spanning geometric coordinate operations (e.g., distance and bearing computation) and reasoning that integrates coordinates with world knowledge. Focusing on intrinsic model capabilities rather than tool use, we evaluate 14 state-of-the-art LLMs and find that GPS reasoning remains challenging, with substantial variation across tasks: models are generally more reliable at real-world geographic reasoning than at geometric computations. Geographic knowledge degrades hierarchically, with strong country-level performance but weak city-level localization, while robustness to coordinate noise suggests genuine coordinate understanding rather than memorization. We further show that GPS-coordinate augmentation can improve in downstream geospatial tasks, and that finetuning induces trade-offs between gains in geometric computation and degradation in world knowledge. Our dataset and reproducible code are available at https://github.com/joey234/gpsbench