mlsanityData to 7 Oct 2026

Toward Reliable VLM: A Fine-Grained Benchmark and Framework for Exposure, Bias, and Inference in Korean Street Views

arXiv 2506.03371 ↗First posted 3 Jun 2025cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Recent advances in vision-language models (VLMs) have enabled accurate image-based geolocation, raising serious concerns about location privacy risks in everyday social media posts. However, current benchmarks remain coarse-grained, linguistically biased, and lack multimodal and privacy-aware evaluations. To address these gaps, we present KoreaGEO Bench, the first fine-grained, multimodal geolocation benchmark for Korean street views. Our dataset comprises 1,080 high-resolution images sampled across four urban clusters and nine place types, enriched with multi-contextual annotations and two styles of Korean captions simulating real-world privacy exposure. We introduce a three-path evaluation protocol to assess ten mainstream VLMs under varying input modalities and analyze their accuracy, spatial bias, and reasoning behavior. Results reveal modality-driven shifts in localization precision and highlight structural prediction biases toward core cities.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 51 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

23 of 51 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025STRIVE: Structured Representation Integrating VLM Reasoning for Efficient Object Navigation10 May
  2. 2025NAVIG: Natural Language-guided Analysis with Vision Language Models for Image Geo-localization20 Feb
  3. 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding27 Jan
  4. 2024LLMGeo: Benchmarking Large Language Models on Image Geolocation In-the-wild30 May

Other references · 19

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  4. 2019Defending Against Neural Fake Newscited by 553
  5. 2015Wide-Area Image Geolocalization with Aerial Reference Imagerycited by 185
  6. 2019Lending Orientation to Neural Networks for Cross-view Geo-localizationcited by 179
  7. 2020University-1652: A Multi-view Multi-source Benchmark for Drone-based Geo-localizationcited by 152
  8. 2016PlaNet - Photo Geolocation with Convolutional Neural Networkscited by 114
  9. 2023GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognitioncited by 50
  10. 2016Automatic Description Generation from Images: A Survey of Models, Datasets, and Evaluation Measurescited by 36
  11. 2024Private Attribute Inference from Images with Vision-Language Modelscited by 35
  12. 2025The Bitter Lesson Learned from 2,000+ Multilingual Benchmarkscited by 31
7 more
  1. 2025Geolocation with Real Human Gameplay Data: A Large-Scale Dataset and Human-Like Reasoning Frameworkcited by 23
  2. 2023Can Vision-Language Models be a Good Guesser? Exploring VLMs for Times and Location Reasoningcited by 22
  3. 2024CV-Cities: Advancing Cross-View Geo-Localization in Global Citiescited by 22
  4. 2024Towards Interactive Global Geolocation Assistantcited by 11
  5. 2024Comparing Traditional and LLM-based Search for Image Geolocationcited by 10
  6. 2024CityGuessr: City-Level Video Geo-Localization on a Global Scalecited by 10
  7. 2018DeepGeo: Photo Localization with Deep Neural Networkcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026GeoContext: One Context Ladder, Two Failure Modes in Vision-Language Geolocation: Flat Reliance on User-Provided Location Context and False Confirmation of Location Claimscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Evaluation of Geolocation Capabilities of Multimodal Large Language Models and Analysis of Associated Privacy Risks
  2. 2025Evaluating Precise Geolocation Inference Capabilities of Vision Language Models
  3. 2025Assessing the Geolocation Capabilities, Limitations and Societal Risks of Generative Vision-Language Models
  4. 2026Do Vision-Language Models Respect Contextual Integrity in Location Disclosure?
  5. 2025From Pixels to Places: A Systematic Benchmark for Evaluating Image Geolocalization Ability in Large Language Models
  6. 2024Granular Privacy Control for Geolocation with Vision Language Models
  7. 2024Image-Based Geolocation Using Large Vision-Language Models
  8. 2025GEO-Detective: Unveiling Location Privacy Risks in Images with LLM Agents