Does Object Recognition Work for Everyone?
Not on the map. Why?
Abstract (arXiv)
The paper analyzes the accuracy of publicly available object-recognition systems on a geographically diverse dataset. This dataset contains household items and was designed to have a more representative geographical coverage than commonly used image datasets in object recognition. We find that the systems perform relatively poorly on household items that commonly occur in countries with a low household income. Qualitative analyses suggest the drop in performance is primarily due to appearance differences within an object class (e.g., dish soap) and due to items appearing in a different context (e.g., toothbrushes appearing outside of bathrooms). The results of our study suggest that further work is needed to make object-recognition systems work equally well for people across different countries and income levels.
Key facts. Cited by 151 papers in the map (data through 14 Oct 2026). First posted on arXiv 6 Jun 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 24; 2022: 42; 2023: 45; 2024: 24; 2025: 10; 2026: 6.
151 papers.
Sort:
By area:
- 2023Segment Anythingcited by 8,234
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
- 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 434
- 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
- 2021AI and the Everything in the Whole Wide World Benchmarkcited by 349
- 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
- 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 255
- 2021Visually Grounded Reasoning across Languages and Culturescited by 189
- 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 180
- 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 156
38 more
- 2022Domain Generalization by Mutual-Information Regularization with Pre-trained Modelscited by 133
- 2022Revisiting Weakly Supervised Pre-Training of Visual Perception Modelscited by 116
- 2021Understanding and Evaluating Racial Biases in Image Captioningcited by 108
- 2021Distilling Object Detectors via Decoupled Featurescited by 106
- 2022A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learningcited by 104
- 2021Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Developmentcited by 85
- 2022A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Otherscited by 85
- 2021Towards Measuring Fairness in AI: the Casual Conversations Datasetcited by 80
- 2022Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervisioncited by 80
- 2023The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AIcited by 73
- 2023The effectiveness of MAE pre-pretraining for billion-scale pretrainingcited by 68
- 2022A Systematic Study of Bias Amplificationcited by 65
- 2025Meta CLIP 2: A Worldwide Scaling Recipecited by 64
- 2021The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Modelscited by 63
- 2021Broaden the Vision: Geo-Diverse Visual Commonsense Reasoningcited by 59
- 2023Discovering and Mitigating Visual Biases through Keyword Explanationcited by 58
- 2023FACET: Fairness in Computer Vision Evaluation Benchmarkcited by 58
- 2022Learning with Neighbor Consistency for Noisy Labelscited by 55
- 2021PASS: An ImageNet replacement for self-supervised pretraining without humanscited by 51
- 2023GeoDE: a Geographically Diverse Evaluation Dataset for Object Recognitioncited by 50
- 2024Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approachcited by 50
- 2021Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffscited by 48
- 2021A Step Toward More Inclusive People Annotations for Fairnesscited by 46
- 2024Consent in Crisis: The Rapid Decline of the AI Data Commonscited by 46
- 2022ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotationscited by 44
- 2023Inspecting the Geographical Representativeness of Images from Text-to-Image Modelscited by 44
- 2022Adaptive Testing of Computer Vision Modelscited by 39
- 2022Gender Artifacts in Visual Datasetscited by 38
- 2022Quantifying Societal Bias Amplification in Image Captioningcited by 37
- 2022Measuring Representational Harms in Image Captioningcited by 36
- 2023Retrieval-Enhanced Contrastive Vision-Text Modelscited by 36
- 2024CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?cited by 35
- 2023ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classificationcited by 33
- 2023Overwriting Pretrained Bias with Finetuning Datacited by 32
- 2022Understanding Machine Learning Practitioners' Data Documentation Perceptions, Needs, Challenges, and Desideratacited by 28
- 2022Toward Fairness in Speech Recognition: Discovery and mitigation of performance disparitiescited by 28
- 2023Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Setscited by 27
- 2022Incidents1M: a large-scale dataset of images with natural disasters, damage, and incidentscited by 26
50 most cited of 151 All 151 (JSON)
- 2026Whose Data, Whose Benchmark? A User-Centric Framework for Auditing Evaluation Resources13 Oct 2026
- 2026Tanawpamana: Benchmarking Vision-Language Models on Philippine Cultural Heritage Sites13 Oct 2026
- 2026AtlasNLP: A Country-Aware Atlas of Dataset Representation in NLP31 Aug 2026
- 2026Position: the Stochastic Parrot in the Coal Mine. Model Collapse is a Threat to Low-Resource Communities5 May 2026
- 2026Who Gets Flagged? The Pluralistic Evaluation Gap in AI Content Watermarking15 Apr 2026
- 2026Where Do Images Come From? Analyzing Captions to Geographically Profile Datasets10 Feb 2026
- 2025Culture Affordance Atlas: Reconciling Object Diversity Through Functional Mapping2 Dec 2025
- 2025Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models4 Oct 2025
- 2025RefAM: Attention Magnets for Zero-Shot Referral Segmentation26 Sep 2025
- 2025Meta CLIP 2: A Worldwide Scaling Recipe29 Jul 2025
- 2025CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systems9 Jun 2025
- 2025Uncovering Cultural Representation Disparities in Vision-Language Models20 May 2025
38 more
- 2025CRAFT: Cultural Russian-Oriented Dataset Adaptation for Focused Text-to-Image Generation7 May 2025
- 2025JEEM: Vision-Language Understanding in Four Arabic Dialects27 Mar 2025
- 2025Attention IoU: Examining Biases in CelebA using Attention Maps25 Mar 2025
- 2025A Critical Review of Predominant Bias in Neural Networks16 Feb 2025
- 2024Classification Drives Geographic Bias in Street Scene Segmentation15 Dec 2024
- 2024EvalGIM: A Library for Evaluating Generative Image Models13 Dec 2024
- 2024Efficient Bias Mitigation Without Privileged Information26 Sep 2024
- 2024Reassessing the Validity of Spurious Correlations Benchmarks6 Sep 2024
- 2024Fairness and Bias Mitigation in Computer Vision: A Survey5 Aug 2024
- 2024Consent in Crisis: The Rapid Decline of the AI Data Commons20 Jul 2024
- 2024Resampled Datasets Are Not Enough: Mitigating Societal Bias Beyond Single Attributes4 Jul 2024
- 2024Uplifting Lower-Income Data: Strategies for Socioeconomic Perspective Shifts in Large Multi-modal Models2 Jul 2024
- 2024CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluation1 Jul 2024
- 2024From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models28 Jun 2024
- 2024See It from My Perspective: How Language Affects Cultural Bias in Image Understanding17 Jun 2024
- 2024Situated Ground Truths: Enhancing Bias-Aware AI by Situating Data Labels with SituAnnotate10 Jun 2024
- 2024Multilingual Diversity Improves Vision-Language Representations27 May 2024
- 2024Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approach24 May 2024
- 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Models22 May 2024
- 2024Towards Geographic Inclusion in the Evaluation of Text-to-Image Models7 May 2024
- 2024Robust 3D Shape Reconstruction in Zero-Shot from a Single Image in the Wild21 Mar 2024
- 2024Annotations on a Budget: Leveraging Geo-Data Similarity to Balance Model Performance and Annotation Cost12 Mar 2024
- 2024CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?7 Mar 2024
- 2024A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models21 Feb 2024
- 2024The Visual Experience Dataset: Over 200 Recorded Hours of Integrated Eye Movement, Odometry, and Egocentric Video15 Feb 2024
- 2024Multi-group Learning for Hierarchical Groups1 Feb 2024
- 2024Digital Divides in Scene Recognition: Uncovering Socioeconomic Biases in Deep Learning Systems23 Jan 2024
- 2024Incorporating Geo-Diverse Knowledge into Prompting for Increased Geographical Robustness in Object Recognition3 Jan 2024
- 2023Stable Rivers: A Case Study in the Application of Text-to-Image Generative Models for Earth Sciences13 Dec 2023
- 2023Multitask Learning Can Improve Worst-Group Outcomes5 Dec 2023
- 2023Classification for everyone : Building geography agnostic models for fairer recognition5 Dec 2023
- 2023Bridging the Digital Divide: Performance Variation across Socio-Economic Factors in Vision-Language Models9 Nov 2023
- 2023The Data Provenance Initiative: A Large Scale Audit of Dataset Licensing & Attribution in AI25 Oct 2023
- 2023On Responsible Machine Learning Datasets with Fairness, Privacy, and Regulatory Norms24 Oct 2023
- 2023Survey of Social Bias in Vision-Language Models24 Sep 2023
- 2023FACET: Fairness in Computer Vision Evaluation Benchmark31 Aug 2023
- 2023Flickr Africa: Examining Geo-Diversity in Large-Scale, Human-Centric Visual Data16 Aug 2023
- 2023DIG In: Evaluating Disparities in Image Generations with Indicators for Geographic Diversity11 Aug 2023
50 newest of 151 All 151 (JSON)
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2022A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learningcited by 104
- 2025Meta CLIP 2: A Worldwide Scaling Recipecited by 64
- 2023Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Setscited by 27
- 2023Survey of Social Bias in Vision-Language Modelscited by 26
- 2024See It from My Perspective: How Language Affects Cultural Bias in Image Understandingcited by 17
- 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Modelscited by 16
- 2024From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Modelscited by 14
- 2023GIVL: Improving Geographical Inclusivity of Vision-Language Models with Pre-Training Methodscited by 11
- 2023Bridging the Digital Divide: Performance Variation across Socio-Economic Factors in Vision-Language Modelscited by 11
- 2024A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Modelscited by 11
- 2024CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluationcited by 8
12 most cited of 16 All 16 (JSON)
- 2026Tanawpamana: Benchmarking Vision-Language Models on Philippine Cultural Heritage Sites13 Oct 2026
- 2025Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models4 Oct 2025
- 2025Meta CLIP 2: A Worldwide Scaling Recipe29 Jul 2025
- 2025Uncovering Cultural Representation Disparities in Vision-Language Models20 May 2025
- 2025JEEM: Vision-Language Understanding in Four Arabic Dialects27 Mar 2025
- 2024CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluation1 Jul 2024
- 2024From Local Concepts to Universals: Evaluating the Multicultural Understanding of Vision-Language Models28 Jun 2024
- 2024See It from My Perspective: How Language Affects Cultural Bias in Image Understanding17 Jun 2024
- 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Models22 May 2024
- 2024A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models21 Feb 2024
- 2023Bridging the Digital Divide: Performance Variation across Socio-Economic Factors in Vision-Language Models9 Nov 2023
- 2023Survey of Social Bias in Vision-Language Models24 Sep 2023
12 newest of 16 All 16 (JSON)
- 2021Distilling Object Detectors via Decoupled Featurescited by 106
- 2021Distilling Object Detectors via Decoupled Features26 Mar 2021
- 2023Segment Anythingcited by 8,234
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
- 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 434
- 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 360
- 2021AI and the Everything in the Whole Wide World Benchmarkcited by 349
- 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
- 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 255
- 2021Visually Grounded Reasoning across Languages and Culturescited by 189
- 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 180
- 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 156
- 2022Domain Generalization by Mutual-Information Regularization with Pre-trained Modelscited by 133
12 most cited of 129 All 129 (JSON)
- 2026Whose Data, Whose Benchmark? A User-Centric Framework for Auditing Evaluation Resources13 Oct 2026
- 2026AtlasNLP: A Country-Aware Atlas of Dataset Representation in NLP31 Aug 2026
- 2026Where Do Images Come From? Analyzing Captions to Geographically Profile Datasets10 Feb 2026
- 2025Culture Affordance Atlas: Reconciling Object Diversity Through Functional Mapping2 Dec 2025
- 2025CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systems9 Jun 2025
- 2025CRAFT: Cultural Russian-Oriented Dataset Adaptation for Focused Text-to-Image Generation7 May 2025
- 2025Attention IoU: Examining Biases in CelebA using Attention Maps25 Mar 2025
- 2025A Critical Review of Predominant Bias in Neural Networks16 Feb 2025
- 2024Classification Drives Geographic Bias in Street Scene Segmentation15 Dec 2024
- 2024EvalGIM: A Library for Evaluating Generative Image Models13 Dec 2024
- 2024Efficient Bias Mitigation Without Privileged Information26 Sep 2024
- 2024Reassessing the Validity of Spurious Correlations Benchmarks6 Sep 2024
12 newest of 129 All 129 (JSON)