# AI Sees Your Location, But With A Bias Toward The Wealthy World

arXiv: https://arxiv.org/abs/2502.11163 · HTML: https://mlsanity.com/paper/2502.11163/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 16 Feb 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Embodied and 3D vision-language models](https://mlsanity.com/mllm/embodied-and-3d-vision-language-models/)

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding](https://mlsanity.com/paper/2501.16411/) (2025)
- [K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilization](https://mlsanity.com/paper/2306.05064/) (2023)
- [GPT4GEO: How a Language Model Sees the World's Geography](https://mlsanity.com/paper/2306.00020/) (2023)
- [Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the US](https://mlsanity.com/paper/1702.06683/) (2017)
- [Measuring Social Biases in Grounded Vision and Language Embeddings](https://mlsanity.com/paper/2002.08911/) (2020)
- [Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Images](https://mlsanity.com/paper/2402.05779/) (2024)
- [BiasAsker: Measuring the Bias in Conversational AI System](https://mlsanity.com/paper/2305.12434/) (2023)
- [VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution](https://mlsanity.com/paper/2306.12424/) (2023)
- [Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Bias](https://mlsanity.com/paper/2212.11261/) (2022)
- [American == White in Multimodal Language-and-Image AI](https://mlsanity.com/paper/2207.00691/) (2022)
- [GeoLM: Empowering Language Models for Geospatially Grounded Language Understanding](https://mlsanity.com/paper/2310.14478/) (2023)
- [Assessment of Multimodal Large Language Models in Alignment with Human Values](https://mlsanity.com/paper/2403.17830/) (2024)
- [Geo-knowledge-guided GPT models improve the extraction of location descriptions from disaster-related social media messages](https://mlsanity.com/paper/2310.09340/) (2023)
- [Are Large Language Models Geospatially Knowledgeable?](https://mlsanity.com/paper/2310.13002/) (2023)
- [Counterfactually Measuring and Eliminating Social Bias in Vision-Language Pre-training Models](https://mlsanity.com/paper/2207.01056/) (2022)
- [Ocean-OCR: Towards General OCR Application via a Vision-Language Model](https://mlsanity.com/paper/2501.15558/) (2025)
- [Multimodal Foundation Models Exploit Text to Make Medical Image Predictions](https://mlsanity.com/paper/2311.05591/) (2023)
- [New Job, New Gender? Measuring the Social Bias in Image Generation Models](https://mlsanity.com/paper/2401.00763/) (2024)
- [GeoLocator: a location-integrated large multimodal model for inferring geo-privacy](https://mlsanity.com/paper/2311.13018/) (2023)
- [Comparing Traditional and LLM-based Search for Image Geolocation](https://mlsanity.com/paper/2401.10184/) (2024)
- [A Multidimensional Analysis of Social Biases in Vision Transformers](https://mlsanity.com/paper/2308.01948/) (2023)
- [The Male CEO and the Female Assistant: Evaluation and Mitigation of Gender Biases in Text-To-Image Generation of Dual Subjects](https://mlsanity.com/paper/2402.11089/) (2024)
- [Where Fact Ends and Fairness Begins: Redefining AI Bias Evaluation through Cognitive Biases](https://mlsanity.com/paper/2502.05849/) (2025)

## Built on by (most cited first, 14 of 14)

- [ReasonMap: Towards Fine-Grained Visual Reasoning from Transit Maps](https://mlsanity.com/paper/2505.18675/) (2025)
- [GeoChain: Multimodal Chain-of-Thought for Geographic Reasoning](https://mlsanity.com/paper/2506.00785/) (2025)
- [Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligence](https://mlsanity.com/paper/2506.15677/) (2025)
- [Thinking with Map: Reinforced Parallel Map-Augmented Agent for Geolocalization](https://mlsanity.com/paper/2601.05432/) (2026)
- [GeoArena: Evaluating Open-World Geographic Reasoning in Large Vision-Language Models](https://mlsanity.com/paper/2509.04334/) (2025)
- [CaughtCheating: Is Your MLLM a Good Cheating Detective? Exploring the Boundary of Visual Perception and Reasoning](https://mlsanity.com/paper/2507.00045/) (2025)
- [FAIRGAMER: Evaluating Social Biases in LLM-Based Video Game NPCs](https://mlsanity.com/paper/2508.17825/) (2025)
- [FlagEval Findings Report: A Preliminary Evaluation of Large Reasoning Models on Automatically Verifiable Textual and Visual Questions](https://mlsanity.com/paper/2509.17177/) (2025)
- [Do Vision-Language Models Respect Contextual Integrity in Location Disclosure?](https://mlsanity.com/paper/2602.05023/) (2026)
- [SIGNPOST-Bench: Benchmarking Text-Vision Conflict Resolution in Multimodal Large Language Models](https://mlsanity.com/paper/2608.04244/) (2026)
- [Controllable Pareto Trade-off between Fairness and Accuracy](https://mlsanity.com/paper/2509.13651/) (2025)
- [GTPred: Benchmarking MLLMs for Interpretable Geo-localization and Time-of-capture Prediction](https://mlsanity.com/paper/2601.13207/) (2026)
- [SDGBiasBench: Benchmarking and Mitigating Vision--Language Models' Biases in Sustainable Development Goals](https://mlsanity.com/paper/2605.21919/) (2026)
- [GeoAgent: Evaluating VLM Geolocalization Through Embodied Navigation](https://mlsanity.com/paper/2608.29483/) (2026)

## Abstract (arXiv)

Visual-Language Models (VLMs) have shown remarkable performance across various tasks, particularly in recognizing geographic information from images. However, VLMs still show regional biases in this task. To systematically evaluate these issues, we introduce a benchmark consisting of 1,200 images paired with detailed geographic metadata. Evaluating four VLMs, we find that while these models demonstrate the ability to recognize geographic information from images, achieving up to 53.8% accuracy in city prediction, they exhibit significant biases. Specifically, performance is substantially higher for economically developed and densely populated regions compared to less developed (-12.5%) and sparsely populated (-17.0%) areas. Moreover, regional biases of frequently over-predicting certain locations remain. For instance, they consistently predict Sydney for images taken in Australia, shown by the low entropy scores for these countries. The strong performance of VLMs also raises privacy concerns, particularly for users who share images online without the intent of being identified. Our code and dataset are publicly available at https://github.com/uscnlp-lime/FairLocator.