# GeoArena: Evaluating Open-World Geographic Reasoning in Large Vision-Language Models

arXiv: https://arxiv.org/abs/2509.04334 · HTML: https://mlsanity.com/paper/2509.04334/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 21 of its 62 references that are in the map.
- First posted on arXiv 4 Sep 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)
- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 21 of 21)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Gemma 3 Technical Report](https://mlsanity.com/paper/2503.19786/) (2025)
- [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://mlsanity.com/paper/2404.04475/) (2024)
- [Gemini: A Family of Highly Capable Multimodal Models](https://mlsanity.com/paper/2312.11805/) (2023)
- [What Large Language Models Know and What People Think They Know](https://mlsanity.com/paper/2401.13835/) (2024)
- [From Single to Multi-Granularity: Toward Long-Term Memory Association and Selection of Conversational Agents](https://mlsanity.com/paper/2505.19549/) (2025)
- [Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Models](https://mlsanity.com/paper/2506.14674/) (2025)
- [LLMGeo: Benchmarking Large Language Models on Image Geolocation In-the-wild](https://mlsanity.com/paper/2405.20363/) (2024)
- [MapEval: A Map-Based Evaluation of Geo-Spatial Reasoning in Foundation Models](https://mlsanity.com/paper/2501.00316/) (2024)
- [GeoRanker: Distance-Aware Ranking for Worldwide Image Geolocalization](https://mlsanity.com/paper/2505.13731/) (2025)
- [UniTraj: Learning a Universal Trajectory Foundation Model from Billion-Scale Worldwide Traces](https://mlsanity.com/paper/2411.03859/) (2024)
- [Image-Based Geolocation Using Large Vision-Language Models](https://mlsanity.com/paper/2408.09474/) (2024)
- [GeoChain: Multimodal Chain-of-Thought for Geographic Reasoning](https://mlsanity.com/paper/2506.00785/) (2025)
- [AI Sees Your Location, But With A Bias Toward The Wealthy World](https://mlsanity.com/paper/2502.11163/) (2025)
- [Visual and Object Geo-localization: A Comprehensive Survey](https://mlsanity.com/paper/2112.15202/) (2021)
- [Evaluating Precise Geolocation Inference Capabilities of Vision Language Models](https://mlsanity.com/paper/2502.14412/) (2025)
- [From Pixels to Places: A Systematic Benchmark for Evaluating Image Geolocalization Ability in Large Language Models](https://mlsanity.com/paper/2508.01608/) (2025)
- [Towards Interactive Global Geolocation Assistant](https://mlsanity.com/paper/2412.08907/) (2024)
- [Computer Vision for Multimedia Geolocation in Human Trafficking Investigation: A Systematic Literature Review](https://mlsanity.com/paper/2402.15448/) (2024)

## Built on by (most cited first, 5 of 5)

- [Thinking with Map: Reinforced Parallel Map-Augmented Agent for Geolocalization](https://mlsanity.com/paper/2601.05432/) (2026)
- [Where on Earth? A Vision-Language Benchmark for Probing Model Geolocation Skills Across Scales](https://mlsanity.com/paper/2510.10880/) (2025)
- [Do Vision-Language Models Respect Contextual Integrity in Location Disclosure?](https://mlsanity.com/paper/2602.05023/) (2026)
- [GeoRouter: Dynamic Paradigm Routing for Worldwide Image Geolocalization](https://mlsanity.com/paper/2603.24376/) (2026)
- [Perceive to Hypothesize, Verify to Ground: An Agentic Reasoning Framework for Open-World Geo-Localization](https://mlsanity.com/paper/2608.29880/) (2026)

## Abstract (arXiv)

Geographic reasoning is a fundamental cognitive capability that requires models to infer plausible locations by synthesizing visual evidence with spatial world knowledge. Despite recent advances in large vision-language models (LVLMs), existing evaluation paradigms remain largely outcome-centric, relying on static datasets and predefined labels that are conceptually misaligned with open-world geographic inference. Such outcome-centric evaluations often focus exclusively on label matching, leaving the underlying linguistic reasoning chains as unexamined black boxes. In this work, we introduce GeoArena, a dynamic, human-preference-based evaluation framework for benchmarking open-world geographic reasoning. GeoArena reframes evaluation as a pairwise reasoning alignment task on in-the-wild images, where human judges compare model-generated explanations based on reasoning quality, evidence synthesis, and plausibility. We deploy GeoArena as a public platform and benchmark 17 frontier LVLMs using thousands of human judgments, which complements existing benchmarks and supports the development of geographically grounded, human-aligned AI systems. We further provide detailed analyses of model behavior, including reliability of human preferences and factors influencing judgments of geographic reasoning quality.