# Where Do Vision-Language Models Fail? World Scale Analysis for Image Geolocalization

arXiv: https://arxiv.org/abs/2604.16248 · HTML: https://mlsanity.com/paper/2604.16248/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 30 of its 42 references that are in the map.
- First posted on arXiv 17 Apr 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://mlsanity.com/paper/2412.05271/) (2024)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questions](https://mlsanity.com/paper/2308.11483/) (2023)
- [End-to-end Learning of Deep Visual Representations for Image Retrieval](https://mlsanity.com/paper/1610.07940/) (2016)
- [Self-supervising Fine-grained Region Similarities for Large-scale Image Localization](https://mlsanity.com/paper/2006.03926/) (2020)
- [Localizing and Orienting Street Views Using Overhead Imagery](https://mlsanity.com/paper/1608.00161/) (2016)
- [Deep Visual Geo-localization Benchmark](https://mlsanity.com/paper/2204.03444/) (2022)
- [Benchmarking Image Retrieval for Visual Localization](https://mlsanity.com/paper/2011.11946/) (2020)
- [Adaptive-Attentive Geolocalization from few queries: a hybrid approach](https://mlsanity.com/paper/2010.06897/) (2020)
- [OpenStreetView-5M: The Many Roads to Global Visual Geolocation](https://mlsanity.com/paper/2404.18873/) (2024)
- [Where am I? Cross-View Geo-localization with Natural Language Descriptions](https://mlsanity.com/paper/2412.17007/) (2024)
- [Charting New Territories: Exploring the Geographic and Geospatial Capabilities of Multimodal LLMs](https://mlsanity.com/paper/2311.14656/) (2023)
- [DenserNet: Weakly Supervised Visual Localization Using Multi-scale Feature Aggregation](https://mlsanity.com/paper/2012.02366/) (2020)
- [Cross-View Image Sequence Geo-localization](https://mlsanity.com/paper/2210.14295/) (2022)
- [Unleashing Unlabeled Data: A Paradigm for Cross-View Geo-Localization](https://mlsanity.com/paper/2403.14198/) (2024)
- [GAMa: Cross-view Video Geo-localization](https://mlsanity.com/paper/2207.02431/) (2022)
- [UAV Pose Estimation using Cross-view Geolocalization with Satellite Imagery](https://mlsanity.com/paper/1809.05979/) (2018)
- [Order Matters: Exploring Order Sensitivity in Multimodal Large Language Models](https://mlsanity.com/paper/2410.16983/) (2024)
- [GRE Suite: Geo-localization Inference via Fine-Tuned Vision-Language Models and Enhanced Reasoning Chains](https://mlsanity.com/paper/2505.18700/) (2025)
- [CityGuessr: City-Level Video Geo-Localization on a Global Scale](https://mlsanity.com/paper/2411.06344/) (2024)
- [Where on Earth? A Vision-Language Benchmark for Probing Model Geolocation Skills Across Scales](https://mlsanity.com/paper/2510.10880/) (2025)
- [GeoX-Bench: Benchmarking Cross-View Geo-Localization and Pose Estimation Capabilities of Large Multimodal Models](https://mlsanity.com/paper/2511.13259/) (2025)
- [GlobalGeoTree: A Multi-Granular Vision-Language Dataset for Global Tree Species Classification](https://mlsanity.com/paper/2505.12513/) (2025)

## Abstract (arXiv)

Image geolocalization has traditionally been addressed through retrieval-based place recognition or geometry-based visual localization pipelines. Recent advances in Vision-Language Models (VLMs) have demonstrated strong zero-shot reasoning capabilities across multimodal tasks, yet their performance in geographic inference remains underexplored. In this work, we present a systematic evaluation of multiple state-of-the-art VLMs for country-level image geolocalization using ground-view imagery only. Instead of relying on image matching, GPS metadata, or task-specific training, we evaluate prompt-based country prediction in a zero-shot setting. The selected models are tested on three geographically diverse datasets to assess their robustness and generalization ability. Our results reveal substantial variation across models, highlighting the potential of semantic reasoning for coarse geolocalization and the limitations of current VLMs in capturing fine-grained geographic cues. This study provides the first focused comparison of modern VLMs for country-level geolocalization and establishes a foundation for future research at the intersection of multimodal reasoning and geographic understanding.