# From Pixels to Places: A Systematic Benchmark for Evaluating Image Geolocalization Ability in Large Language Models

arXiv: https://arxiv.org/abs/2508.01608 · HTML: https://mlsanity.com/paper/2508.01608/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 3 Aug 2025.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval](https://mlsanity.com/paper/2011.12172/) (2020)
- [GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localization](https://mlsanity.com/paper/2309.16020/) (2023)
- [Wide-Area Image Geolocalization with Aerial Reference Imagery](https://mlsanity.com/paper/1510.03743/) (2015)
- [TransGeo: Transformer Is All You Need for Cross-view Image Geo-localization](https://mlsanity.com/paper/2204.00097/) (2022)
- [GEO-Bench: Toward Foundation Models for Earth Monitoring](https://mlsanity.com/paper/2306.03831/) (2023)
- [Revisiting IM2GPS in the Deep Learning Era](https://mlsanity.com/paper/1705.04838/) (2017)
- [Cross-View Image Matching for Geo-localization in Urban Environments](https://mlsanity.com/paper/1703.07815/) (2017)
- [CPlaNet: Enhancing Image Geolocalization by Combinatorial Partitioning of Maps](https://mlsanity.com/paper/1808.02130/) (2018)
- [Where in the World is this Image? Transformer-based Geo-localization in the Wild](https://mlsanity.com/paper/2204.13861/) (2022)
- [Where We Are and What We're Looking At: Query Based Worldwide Image Geo-localization Using Hierarchies and Scenes](https://mlsanity.com/paper/2303.04249/) (2023)
- [NuScenes-SpatialQA: A Spatial Understanding and Reasoning Benchmark for Vision-Language Models in Autonomous Driving](https://mlsanity.com/paper/2504.03164/) (2025)
- [Personalized Showcases: Generating Multi-Modal Explanations for Recommendations](https://mlsanity.com/paper/2207.00422/) (2022)
- [Enhancing Visual Document Understanding with Contrastive Learning in Large Visual-Language Models](https://mlsanity.com/paper/2402.19014/) (2024)
- [Cross-View Geolocalization and Disaster Mapping with Street-View and VHR Satellite Imagery: A Case Study of Hurricane IAN](https://mlsanity.com/paper/2408.06761/) (2024)
- [Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities](https://mlsanity.com/paper/2406.02787/) (2024)
- [LLMGeo: Benchmarking Large Language Models on Image Geolocation In-the-wild](https://mlsanity.com/paper/2405.20363/) (2024)
- [AddressCLIP: Empowering Vision-Language Models for City-wide Image Address Localization](https://mlsanity.com/paper/2407.08156/) (2024)
- [You Are Here: Geolocation by Embedding Maps and Images](https://mlsanity.com/paper/1911.08797/) (2019)
- [GT-Loc: Unifying When and Where in Images Through a Joint Embedding Space](https://mlsanity.com/paper/2507.10473/) (2025)
- [GeoLocSFT: Efficient Visual Geolocation via Supervised Fine-Tuning of Multimodal Foundation Models](https://mlsanity.com/paper/2506.01277/) (2025)
- [LLMs as World Models: Data-Driven and Human-Centered Pre-Event Simulation for Disaster Impact Assessment](https://mlsanity.com/paper/2506.06355/) (2025)

## Built on by (most cited first, 12 of 12)

- [Empowering LLM Agents with Geospatial Awareness: Toward Grounded Reasoning for Wildfire Response](https://mlsanity.com/paper/2510.12061/) (2025)
- [Thinking with Map: Reinforced Parallel Map-Augmented Agent for Geolocalization](https://mlsanity.com/paper/2601.05432/) (2026)
- [GeoArena: Evaluating Open-World Geographic Reasoning in Large Vision-Language Models](https://mlsanity.com/paper/2509.04334/) (2025)
- [Unlocking Zero-Shot Geospatial Reasoning via Indirect Rewards](https://mlsanity.com/paper/2510.00072/) (2025)
- [Locatability-Guided Adaptive Reasoning for Image Geo-Localization with Vision-Language Models](https://mlsanity.com/paper/2603.13628/) (2026)
- [LocationAgent: A Hierarchical Agent for Image Geolocation via Decoupling Strategy and Evidence from Parametric Knowledge](https://mlsanity.com/paper/2601.19155/) (2026)
- [HoloGeo: Mitigating Landmark Bias in Geo-localization via Evidence-Driven Reasoning](https://mlsanity.com/paper/2607.15255/) (2026)
- [TimeSpot: Benchmarking Geo-Temporal Understanding in Vision-Language Models in Real-World Settings](https://mlsanity.com/paper/2603.06687/) (2026)
- [GeoRouter: Dynamic Paradigm Routing for Worldwide Image Geolocalization](https://mlsanity.com/paper/2603.24376/) (2026)
- [ERGeoBench:A Comprehensive Benchmark for Embodied Reasoning and Geo-localization in Multimodal Large Language Models](https://mlsanity.com/paper/2605.31251/) (2026)
- [GeoContext: One Context Ladder, Two Failure Modes in Vision-Language Geolocation: Flat Reliance on User-Provided Location Context and False Confirmation of Location Claims](https://mlsanity.com/paper/2609.05761/) (2026)
- [GeoCloak: An Agentic Framework for Semantics-Preserving Geolocation Privacy via Peripheral Clue Manipulation](https://mlsanity.com/paper/2610.20055/) (2026)

## Abstract (arXiv)

Image geolocalization, the task of identifying the geographic location depicted in an image, is important for applications in crisis response, digital forensics, and location-based intelligence. While recent advances in large language models (LLMs) offer new opportunities for visual reasoning, their ability to perform image geolocalization remains underexplored. In this study, we introduce a benchmark called IMAGEO-Bench that systematically evaluates accuracy, distance error, geospatial bias, and reasoning process. Our benchmark includes three diverse datasets covering global street scenes, points of interest (POIs) in the United States, and a private collection of unseen images. Through experiments on 10 state-of-the-art LLMs, including both open- and closed-source models, we reveal clear performance disparities, with closed-source models generally showing stronger reasoning. Importantly, we uncover geospatial biases as LLMs tend to perform better in high-resource regions (e.g., North America, Western Europe, and California) while exhibiting degraded performance in underrepresented areas. Regression diagnostics demonstrate that successful geolocalization is primarily dependent on recognizing urban settings, outdoor environments, street-level imagery, and identifiable landmarks. Overall, IMAGEO-Bench provides a rigorous lens into the spatial reasoning capabilities of LLMs and offers implications for building geolocation-aware AI systems.