mlsanityData to 14 Oct 2026

LLMGeo: Benchmarking Large Language Models on Image Geolocation In-the-wild

arXiv 2405.20363 ↗First posted 30 May 2024cs.CVcited by 21

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Image geolocation is a critical task in various image-understanding applications. However, existing methods often fail when analyzing challenging, in-the-wild images. Inspired by the exceptional background knowledge of multimodal language models, we systematically evaluate their geolocation capabilities using a novel image dataset and a comprehensive evaluation framework. We first collect images from various countries via Google Street View. Then, we conduct training-free and training-based evaluations on closed-source and open-source multi-modal language models. we conduct both training-free and training-based evaluations on closed-source and open-source multimodal language models. Our findings indicate that closed-source models demonstrate superior geolocation abilities, while open-source models can achieve comparable performance through fine-tuning.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 13 of its 17 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

13 of 17 references.

References · 13

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  7. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  8. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  9. 2023GeoCLIP: Clip-Inspired Alignment between Locations and Images for Effective Worldwide Geo-localizationcited by 187
  10. 2023PIGEON: Predicting Image Geolocationscited by 56
  11. 2022Visual Commonsense in Pretrained Unimodal and Multimodal Modelscited by 39
  12. 2023Large Language Models Are Zero-Shot Text Classifierscited by 35
1 more
  1. 2018DeepGeo: Photo Localization with Deep Neural Networkcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 12; 2026: 8.

21 papers.

Sort:

By area:

  1. 2025GeoRanker: Distance-Aware Ranking for Worldwide Image Geolocalizationcited by 19
  2. 2025Evaluating Precise Geolocation Inference Capabilities of Vision Language Modelscited by 12
  3. 2025From Pixels to Places: A Systematic Benchmark for Evaluating Image Geolocalization Ability in Large Language Modelscited by 12
  4. 2026Thinking with Map: Reinforced Parallel Map-Augmented Agent for Geolocalizationcited by 9
  5. 2025GeoShield: Safeguarding Geolocation Privacy from Vision-Language Models via Adversarial Perturbationscited by 5
  6. 2025GeoArena: Evaluating Open-World Geographic Reasoning in Large Vision-Language Modelscited by 5
  7. 2025GeoX-Bench: Benchmarking Cross-View Geo-Localization and Pose Estimation Capabilities of Large Multimodal Modelscited by 5
  8. 2024Benchmarking Large Language Models for Image Classification of Marine Mammalscited by 4
  9. 2025Evaluation of Geolocation Capabilities of Multimodal Large Language Models and Analysis of Associated Privacy Riskscited by 3
  10. 2025Urban Computing in the Era of Large Language Modelscited by 2
  11. 2025Toward Reliable VLM: A Fine-Grained Benchmark and Framework for Exposure, Bias, and Inference in Korean Street Viewscited by 1
  12. 2025GEO-Detective: Unveiling Location Privacy Risks in Images with LLM Agentscited by 1
9 more
  1. 2025Enhancing Geo-localization for Crowdsourced Flood Imagery via LLM-Guided Attentioncited by 1
  2. 2025Beyond Memorization: A Multi-Modal Ordinal Regression Benchmark to Expose Popularity Bias in Vision-Language Modelscited by 1
  3. 2026LocationAgent: A Hierarchical Agent for Image Geolocation via Decoupling Strategy and Evidence from Parametric Knowledgecited by 1
  4. 2026Where Do Images Come From? Analyzing Captions to Geographically Profile Datasetscited by 1
  5. 2026GTPred: Benchmarking MLLMs for Interpretable Geo-localization and Time-of-capture Predictioncited by 0
  6. 2026TimeSpot: Benchmarking Geo-Temporal Understanding in Vision-Language Models in Real-World Settingscited by 0
  7. 2026DamageArbiter: A Multimodal Arbitration Framework for Disaster Damage Assessment from Street-View Imagerycited by 0
  8. 2026GeoRouter: Dynamic Paradigm Routing for Worldwide Image Geolocalizationcited by 0
  9. 2026Hiding in Plain Sight: A Diffusion-based Mitigation of Geolocation Privacy Leakage in Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Where on Earth? A Vision-Language Benchmark for Probing Model Geolocation Skills Across Scales
  2. 2025Assessing the Geolocation Capabilities, Limitations and Societal Risks of Generative Vision-Language Models
  3. 2026Where Do Vision-Language Models Fail? World Scale Analysis for Image Geolocalization
  4. 2024Image-Based Geolocation Using Large Vision-Language Models
  5. 2025Street-Level Geolocalization Using Multimodal Large Language Models and Retrieval-Augmented Generation
  6. 2024Towards Interactive Global Geolocation Assistant
  7. 2026GeoContext: One Context Ladder, Two Failure Modes in Vision-Language Geolocation: Flat Reliance on User-Provided Location Context and False Confirmation of Location Claims
  8. 2026GeoSearch: Augmenting Worldwide Geolocalization with Web-Scale Reverse Image Search and Image Matching