mlsanityData to 14 Oct 2026

Bridging the Digital Divide: Performance Variation across Socio-Economic Factors in Vision-Language Models

arXiv 2311.05746 ↗First posted 9 Nov 2023cs.CY cs.AI cs.CLcited by 11

On the map: Multimodal LLMs / vision-language models → Social bias and fairness in VLMs

Abstract (arXiv)

Despite the impressive performance of current AI models reported across various tasks, performance reports often do not include evaluations of how these models perform on the specific groups that will be impacted by these technologies. Among the minority groups under-represented in AI, data from low-income households are often overlooked in data collection and model evaluation. We evaluate the performance of a state-of-the-art vision-language model (CLIP) on a geo-diverse dataset containing household images associated with different income values (Dollar Street) and show that performance inequality exists among households of different income levels. Our results indicate that performance for the poorer groups is consistently lower than the wealthier groups across various topics and countries. We highlight insights that can help mitigate these issues and propose actionable steps for economic-level inclusive AI development. Code is available at https://github.com/MichiganNLP/Bridging_the_Digital_Divide.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Social bias and fairness in VLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 57 references that are in the map. First posted on arXiv 9 Nov 2023.

Builds on

30 of 57 references.

References · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2023Segment Anythingcited by 8,234
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  7. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  8. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  9. 2018Datasheets for Datasetscited by 1,386
  10. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  11. 2018Model Cards for Model Reportingcited by 1,080
  12. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
18 more
  1. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  2. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529
  3. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 424
  4. 2017No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing Worldcited by 153
  5. 2019Does Object Recognition Work for Everyone?cited by 151
  6. 2020REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasetscited by 136
  7. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 120
  8. 2017Using Deep Learning and Google Street View to Estimate the Demographic Makeup of the UScited by 66
  9. 2021Adaptive Methods for Real-World Domain Generalizationcited by 66
  10. 2020IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLPcited by 58
  11. 2015Transfer Learning from Deep Features for Remote Sensing and Poverty Mappingcited by 37
  12. 2023ChatGPT Needs SPADE (Sustainability, PrivAcy, Digital divide, and Ethics) Evaluation: A Reviewcited by 28
  13. 2022When does Bias Transfer in Transfer Learning?cited by 23
  14. 2022Evaluating the Diversity, Equity and Inclusion of NLP Technology: A Case Study for Indian Languagescited by 20
  15. 2021The Limits of Global Inclusion in AI Developmentcited by 14
  16. 2022This is the way: designing and compiling LEPISZCZE, a comprehensive NLP benchmark for Polishcited by 13
  17. 2021What Vision-Language Models `See' when they See Scenescited by 8
  18. 2019Identifying Visible Actions in Lifestyle Vlogscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 4; 2026: 1.

11 papers.

  1. 2023Eliciting In-Context Learning in Vision-Language Models for Videos Through Curated Data Distributional Propertiescited by 21
  2. 2024See It from My Perspective: How Language Affects Cultural Bias in Image Understandingcited by 17
  3. 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Modelscited by 16
  4. 2024CROPE: Evaluating In-Context Adaptation of Vision and Language Models to Culture-Specific Conceptscited by 11
  5. 2024Annotations on a Budget: Leveraging Geo-Data Similarity to Balance Model Performance and Annotation Costcited by 5
  6. 2025Uncovering Cultural Representation Disparities in Vision-Language Modelscited by 4
  7. 2024Uplifting Lower-Income Data: Strategies for Socioeconomic Perspective Shifts in Large Multi-modal Modelscited by 2
  8. 2025Cultural Awareness in Vision-Language Models: A Cross-Country Explorationcited by 2
  9. 2025From Charts to Fair Narratives: Uncovering and Mitigating Geo-Economic Biases in Chart-to-Textcited by 2
  10. 2025Text-to-Image Models and Their Representation of People from Different Nationalities Engaging in Activitiescited by 0
  11. 2026AtlasNLP: A Country-Aware Atlas of Dataset Representation in NLPcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Unveiling the "Fairness Seesaw": Discovering and Mitigating Gender and Race Bias in Vision-Language Models
  2. 2024A Unified Framework and Dataset for Assessing Societal Bias in Vision-Language Models
  3. 2025Platonic Representations for Poverty Mapping: Unified Vision-Language Codes or Agent-Induced Novelty?
  4. 2024VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model
  5. 2024Dataset Scale and Societal Consistency Mediate Facial Impression Bias in Vision-Language AI
  6. 2024AI-generated faces influence gender stereotypes and racial homogenization
  7. 2024You Never Know: Quantization Induces Inconsistent Biases in Vision-Language Foundation Models
  8. 2026Measuring Social Bias in Vision-Language Models with Face-Only Counterfactuals from Real Photos