# GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognition

arXiv: https://arxiv.org/abs/2301.02560 · HTML: https://mlsanity.com/paper/2301.02560/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 5 Jan 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding](https://mlsanity.com/paper/1604.01753/) (2016)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [Large image datasets: A pyrrhic win for computer vision?](https://mlsanity.com/paper/2006.16923/) (2020)
- [Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy](https://mlsanity.com/paper/1912.07726/) (2019)
- [No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World](https://mlsanity.com/paper/1711.08536/) (2017)
- [Does Object Recognition Work for Everyone?](https://mlsanity.com/paper/1906.02659/) (2019)
- [REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets](https://mlsanity.com/paper/2004.07999/) (2020)
- [Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning](https://mlsanity.com/paper/1912.10389/) (2019)
- [A Study of Face Obfuscation in ImageNet](https://mlsanity.com/paper/2103.06191/) (2021)
- [Adaptive Methods for Real-World Domain Generalization](https://mlsanity.com/paper/2103.15796/) (2021)
- [PASS: An ImageNet replacement for self-supervised pretraining without humans](https://mlsanity.com/paper/2109.13228/) (2021)
- [Auditing ImageNet: Towards a Model-driven Framework for Annotating Demographic Attributes of Large-Scale Image Datasets](https://mlsanity.com/paper/1905.01347/) (2019)

## Built on by (most cited first, 30 of 50)

- [Meta CLIP 2: A Worldwide Scaling Recipe](https://mlsanity.com/paper/2507.22062/) (2025)
- [MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models](https://mlsanity.com/paper/2410.08182/) (2024)
- [Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.08826/) (2025)
- [Consistency-diversity-realism Pareto fronts of conditional image generative models](https://mlsanity.com/paper/2406.10429/) (2024)
- [V-IRL: Grounding Virtual Intelligence in Real Life](https://mlsanity.com/paper/2402.03310/) (2024)
- [African or European Swallow? Benchmarking Large Vision-Language Models for Fine-Grained Object Classification](https://mlsanity.com/paper/2406.14496/) (2024)
- [Position: Measure Dataset Diversity, Don't Just Claim It](https://mlsanity.com/paper/2407.08188/) (2024)
- [Evaluating Visual and Cultural Interpretation: The K-Viscuit Benchmark with Human-VLM Collaboration](https://mlsanity.com/paper/2406.16469/) (2024)
- [Active Data Curation Effectively Distills Large-Scale Multimodal Models](https://mlsanity.com/paper/2411.18674/) (2024)
- [Scaling Pre-training to One Hundred Billion Data for Vision Language Models](https://mlsanity.com/paper/2502.07617/) (2025)
- [Ethical Considerations for Responsible Data Curation](https://mlsanity.com/paper/2302.03629/) (2023)
- [PerceptionCLIP: Visual Classification by Inferring and Conditioning on Contexts](https://mlsanity.com/paper/2308.01313/) (2023)
- [No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Models](https://mlsanity.com/paper/2405.13777/) (2024)
- [Probabilistic Language-Image Pre-Training](https://mlsanity.com/paper/2410.18857/) (2024)
- [Grounding Multilingual Multimodal LLMs With Cultural Knowledge](https://mlsanity.com/paper/2508.07414/) (2025)
- [Multilingual Diversity Improves Vision-Language Representations](https://mlsanity.com/paper/2405.16915/) (2024)
- [From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models](https://mlsanity.com/paper/2407.00263/) (2024)
- [SPARKE: Scalable Prompt-Aware Diversity and Novelty Guidance in Diffusion Models via RKE Score](https://mlsanity.com/paper/2506.10173/) (2025)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [DMLR: Data-centric Machine Learning Research -- Past, Present and Future](https://mlsanity.com/paper/2311.13028/) (2023)
- [Recursive Inference Scaling: A Winning Path to Scalable Inference in Language and Multimodal Systems](https://mlsanity.com/paper/2502.07503/) (2025)
- [Culture in Action: Evaluating Text-to-Image Models through Social Activities](https://mlsanity.com/paper/2511.05681/) (2025)
- [DIG In: Evaluating Disparities in Image Generations with Indicators for Geographic Diversity](https://mlsanity.com/paper/2308.06198/) (2023)
- [The Power of Many: Multi-Agent Multimodal Models for Cultural Image Captioning](https://mlsanity.com/paper/2411.11758/) (2024)
- [Whose Truth? Pluralistic Geo-Alignment for (Agentic) AI](https://mlsanity.com/paper/2508.05432/) (2025)
- [Increasing the Utility of Synthetic Images through Chamfer Guidance](https://mlsanity.com/paper/2508.10631/) (2025)
- [EvalGIM: A Library for Evaluating Generative Image Models](https://mlsanity.com/paper/2412.10604/) (2024)
- [GAEA: A Geolocation Aware Conversational Assistant](https://mlsanity.com/paper/2503.16423/) (2025)
- [Alif: Advancing Urdu Large Language Models via Multilingual Synthetic Data Distillation](https://mlsanity.com/paper/2510.09051/) (2025)
- [Incorporating Geo-Diverse Knowledge into Prompting for Increased Geographical Robustness in Object Recognition](https://mlsanity.com/paper/2401.01482/) (2024)

## Abstract (arXiv)

Current dataset collection methods typically scrape large amounts of data from the web. While this technique is extremely scalable, data collected in this way tends to reinforce stereotypical biases, can contain personally identifiable information, and typically originates from Europe and North America. In this work, we rethink the dataset collection paradigm and introduce GeoDE, a geographically diverse dataset with 61,940 images from 40 classes and 6 world regions, with no personally identifiable information, collected by soliciting images from people around the world. We analyse GeoDE to understand differences in images collected in this manner compared to web-scraping. We demonstrate its use as both an evaluation and training dataset, allowing us to highlight and begin to mitigate the shortcomings in current models, despite GeoDE's relatively small size. We release the full dataset and code at https://geodiverse-data-collection.cs.princeton.edu