# GeoBS: Information-Theoretic Quantification of Geographic Bias in AI Models

arXiv: https://arxiv.org/abs/2509.23482 · HTML: https://mlsanity.com/paper/2509.23482/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 59 references that are in the map.
- First posted on arXiv 27 Sep 2025.

## Builds on (most cited first, 28 of 28)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [StereoSet: Measuring stereotypical bias in pretrained language models](https://mlsanity.com/paper/2004.09456/) (2020)
- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [BBQ: A Hand-Built Bias Benchmark for Question Answering](https://mlsanity.com/paper/2110.08193/) (2021)
- [Bias and Fairness in Large Language Models: A Survey](https://mlsanity.com/paper/2309.00770/) (2023)
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://mlsanity.com/paper/1909.01326/) (2019)
- [BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation](https://mlsanity.com/paper/2101.11718/) (2021)
- [SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2207.08051/) (2022)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Functional Map of the World](https://mlsanity.com/paper/1711.07846/) (2017)
- [CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders](https://mlsanity.com/paper/2311.00566/) (2023)
- [Fairness in Machine Learning: Lessons from Political Philosophy](https://mlsanity.com/paper/1712.03586/) (2017)
- [Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning](https://mlsanity.com/paper/1912.10389/) (2019)
- [Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases](https://mlsanity.com/paper/2010.15052/) (2020)
- [Large Language Models are Geographically Biased](https://mlsanity.com/paper/2402.02680/) (2024)
- [Multi-Scale Representation Learning for Spatial Feature Distributions using Grid Cells](https://mlsanity.com/paper/2003.00824/) (2020)
- [A Generalizable and Accessible Approach to Machine Learning with Global Satellite Imagery](https://mlsanity.com/paper/2010.08168/) (2020)
- [SensitiveLoss: Improving Accuracy and Fairness of Face Representations with Discrimination-Aware Deep Learning](https://mlsanity.com/paper/2004.11246/) (2020)
- [Open High-Resolution Satellite Imagery: The WorldStrat Dataset -- With Application to Super-Resolution](https://mlsanity.com/paper/2207.06418/) (2022)
- [Sphere2Vec: A General-Purpose Location Representation Learning over a Spherical Surface for Large-Scale Geospatial Predictions](https://mlsanity.com/paper/2306.17624/) (2023)
- [TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning](https://mlsanity.com/paper/2406.15658/) (2024)
- [Geographic and Geopolitical Biases of Language Models](https://mlsanity.com/paper/2212.10408/) (2022)
- [ADEPT: A DEbiasing PrompT Framework](https://mlsanity.com/paper/2211.05414/) (2022)
- [Probing the Information Theoretical Roots of Spatial Dependence Measures](https://mlsanity.com/paper/2405.18459/) (2024)

## Built on by (most cited first, 3 of 3)

- [Geography According to ChatGPT -- How Generative AI Represents and Reasons about Geography](https://mlsanity.com/paper/2603.18881/) (2026)
- [Quantifying geographic domain shift to decouple the geospatial transferability of human mobility flow generation models](https://mlsanity.com/paper/2608.21567/) (2026)
- [FairRSFM: A Biome-Aware Benchmark and Debiasing Framework for Remote Sensing Foundation Models](https://mlsanity.com/paper/2610.05790/) (2026)

## Abstract (arXiv)

The widespread adoption of AI models, especially foundation models (FMs), has made a profound impact on numerous domains. However, it also raises significant ethical concerns, including bias issues. Although numerous efforts have been made to quantify and mitigate social bias in AI models, geographic bias (in short, geo-bias) receives much less attention, which presents unique challenges. While previous work has explored ways to quantify geo-bias, these measures are model-specific (e.g., mean absolute deviation of LLM ratings) or spatially implicit (e.g., average fairness scores of all spatial partitions). We lack a model-agnostic, universally applicable, and spatially explicit geo-bias evaluation framework that allows researchers to fairly compare the geo-bias of different AI models and to understand what spatial factors contribute to the geo-bias. In this paper, we establish an information-theoretic framework for geo-bias evaluation, called GeoBS (Geo-Bias Scores). We demonstrate the generalizability of the proposed framework by showing how to interpret and analyze existing geo-bias measures under this framework. Then, we propose three novel geo-bias scores that explicitly take intricate spatial factors (multi-scalability, distance decay, and anisotropy) into consideration. Finally, we conduct extensive experiments on 3 tasks, 8 datasets, and 8 models to demonstrate that both task-specific GeoAI models and general-purpose foundation models may suffer from various types of geo-bias. This framework will not only advance the technical understanding of geographic bias but will also establish a foundation for integrating spatial fairness into the design, deployment, and evaluation of AI systems.