mlsanityData to 14 Oct 2026

AI Sees Your Location, But With A Bias Toward The Wealthy World

arXiv 2502.11163 ↗First posted 16 Feb 2025cs.CV cs.CLcited by 14

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Visual-Language Models (VLMs) have shown remarkable performance across various tasks, particularly in recognizing geographic information from images. However, VLMs still show regional biases in this task. To systematically evaluate these issues, we introduce a benchmark consisting of 1,200 images paired with detailed geographic metadata. Evaluating four VLMs, we find that while these models demonstrate the ability to recognize geographic information from images, achieving up to 53.8% accuracy in city prediction, they exhibit significant biases. Specifically, performance is substantially higher for economically developed and densely populated regions compared to less developed (-12.5%) and sparsely populated (-17.0%) areas. Moreover, regional biases of frequently over-predicting certain locations remain. For instance, they consistently predict Sydney for images taken in Australia, shown by the low entropy scores for these countries. The strong performance of VLMs also raises privacy concerns, particularly for users who share images online without the intent of being identified. Our code and dataset are publicly available at https://github.com/uscnlp-lime/FairLocator.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 26 of its 50 references that are in the map. First posted on arXiv 16 Feb 2025.

Builds on

26 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding27 Jan

Other references · 25

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  4. 2023K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilizationcited by 69
  5. 2023GPT4GEO: How a Language Model Sees the World's Geographycited by 68
  6. 2017Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the UScited by 66
  7. 2020Measuring Social Biases in Grounded Vision and Language Embeddingscited by 61
  8. 2024Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Imagescited by 55
  9. 2023BiasAsker: Measuring the Bias in Conversational AI Systemcited by 53
  10. 2023VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolutioncited by 52
  11. 2022Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Biascited by 43
  12. 2022American == White in Multimodal Language-and-Image AIcited by 39
13 more
  1. 2023GeoLM: Empowering Language Models for Geospatially Grounded Language Understandingcited by 37
  2. 2024Assessment of Multimodal Large Language Models in Alignment with Human Valuescited by 36
  3. 2023Geo-knowledge-guided GPT models improve the extraction of location descriptions from disaster-related social media messagescited by 31
  4. 2023Are Large Language Models Geospatially Knowledgeable?cited by 30
  5. 2022Counterfactually Measuring and Eliminating Social Bias in Vision-Language Pre-training Modelscited by 28
  6. 2025Ocean-OCR: Towards General OCR Application via a Vision-Language Modelcited by 25
  7. 2023Multimodal Foundation Models Exploit Text to Make Medical Image Predictionscited by 20
  8. 2024New Job, New Gender? Measuring the Social Bias in Image Generation Modelscited by 16
  9. 2023GeoLocator: a location-integrated large multimodal model for inferring geo-privacycited by 13
  10. 2024Comparing Traditional and LLM-based Search for Image Geolocationcited by 10
  11. 2023A Multidimensional Analysis of Social Biases in Vision Transformerscited by 8
  12. 2024The Male CEO and the Female Assistant: Evaluation and Mitigation of Gender Biases in Text-To-Image Generation of Dual Subjectscited by 6
  13. 2025Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biasescited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 8; 2026: 6.

14 papers.

Sort:

By area:

  1. 2025ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Mapscited by 23
  2. 2025GeoChain: Multimodal Chain-of-Thought for Geographic Reasoningcited by 16
  3. 2025Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligencecited by 15
  4. 2026Thinking with Map: Reinforced Parallel Map-Augmented Agent for Geolocalizationcited by 9
  5. 2025GeoArena: Evaluating Open-World Geographic Reasoning in Large Vision-Language Modelscited by 5
  6. 2025CaughtCheating: Is Your MLLM a Good Cheating Detective? Exploring the Boundary of Visual Perception and Reasoningcited by 4
  7. 2025FAIRGAMER: Evaluating Social Biases in LLM-Based Video Game NPCscited by 1
  8. 2025FlagEval Findings Report: A Preliminary Evaluation of Large Reasoning Models on Automatically Verifiable Textual and Visual Questionscited by 1
  9. 2026Do Vision-Language Models Respect Contextual Integrity in Location Disclosure?cited by 1
  10. 2026SIGNPOST-Bench: Benchmarking Text-Vision Conflict Resolution in Multimodal Large Language Modelscited by 1
  11. 2025Controllable Pareto Trade-off between Fairness and Accuracycited by 0
  12. 2026GTPred: Benchmarking MLLMs for Interpretable Geo-localization and Time-of-capture Predictioncited by 0
2 more
  1. 2026SDGBiasBench: Benchmarking and Mitigating Vision--Language Models' Biases in Sustainable Development Goalscited by 0
  2. 2026GeoAgent: Evaluating VLM Geolocalization Through Embodied Navigationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Assessing the Geolocation Capabilities, Limitations and Societal Risks of Generative Vision-Language Models
  2. 2025Evaluation of Geolocation Capabilities of Multimodal Large Language Models and Analysis of Associated Privacy Risks
  3. 2025Evaluating Precise Geolocation Inference Capabilities of Vision Language Models
  4. 2025From Pixels to Places: A Systematic Benchmark for Evaluating Image Geolocalization Ability in Large Language Models
  5. 2026Where Do Vision-Language Models Fail? World Scale Analysis for Image Geolocalization
  6. 2024GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks
  7. 2024Image-Based Geolocation Using Large Vision-Language Models
  8. 2023Can Vision-Language Models be a Good Guesser? Exploring VLMs for Times and Location Reasoning