mlsanityData to 7 Oct 2026

Measuring Social Biases in Grounded Vision and Language Embeddings

arXiv 2002.08911 ↗First posted 20 Feb 2020cs.CL cs.AIcited by 61

Not on the map. Why?

Abstract (arXiv)

We generalize the notion of social biases from language embeddings to grounded vision and language embeddings. Biases are present in grounded embeddings, and indeed seem to be equally or more significant than for ungrounded embeddings. This is despite the fact that vision and language can suffer from different biases, which one might hope could attenuate the biases in both. Multiple ways exist to generalize metrics measuring bias in word embeddings to this new setting. We introduce the space of generalizations (Grounded-WEAT and Grounded-SEAT) and demonstrate that three generalizations answer different yet important questions about how biases, language, and vision interact. These metrics are used on a new dataset, the first for grounded bias, created by augmenting extending standard linguistic bias benchmarks with 10,228 images from COCO, Conceptual Captions, and Google Images. Dataset construction is challenging because vision datasets are themselves very biased. The presence of these biases in systems will begin to have real-world consequences as they are deployed, making carefully measuring bias and then mitigating it critical to building a fair society.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 12; 2023: 16; 2024: 15; 2025: 8; 2026: 5.

61 papers.

Sort:

By area:

  1. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  2. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  3. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 203
  4. 2021CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Modelscited by 182
  5. 2024An Introduction to Vision-Language Modelingcited by 175
  6. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  7. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 143
  8. 2022CM3: A Causal Masked Multimodal Model of the Internetcited by 142
  9. 2022ReCLIP: A Strong Zero-Shot Baseline for Referring Expression Comprehensioncited by 119
  10. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  11. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  12. 2022Debiased Contrastive Learning of Unsupervised Sentence Representationscited by 56
38 more
  1. 2023Uncurated Image-Text Datasets: Shedding Light on Demographic Biascited by 51
  2. 2023Modular Visual Question Answering via Code Generationcited by 47
  3. 2023Inspecting the Geographical Representativeness of Images from Text-to-Image Modelscited by 44
  4. 2024DiP-GO: A Diffusion Pruner via Few-step Gradient Optimizationcited by 43
  5. 2021Assessing Multilingual Fairness in Pre-trained Multimodal Representationscited by 35
  6. 2023Vision-Language Models Performing Zero-Shot Tasks Exhibit Gender-based Disparitiescited by 31
  7. 2023On the Independence of Association Bias and Empirical Fairness in Language Modelscited by 29
  8. 2022Counterfactually Measuring and Eliminating Social Bias in Vision-Language Pre-training Modelscited by 28
  9. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27
  10. 2023Survey of Social Bias in Vision-Language Modelscited by 26
  11. 2024Would Deep Generative Models Amplify Bias in Future Models?cited by 24
  12. 2023MultiModal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision Language Modelscited by 23
  13. 2022SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrievalcited by 21
  14. 2024Distribution-Aware Data Expansion with Diffusion Modelscited by 20
  15. 2024Fairness and Bias in Multimodal AI: A Surveycited by 19
  16. 2024SANER: Annotation-free Societal Attribute Neutralizer for Debiasing CLIPcited by 19
  17. 2023Model-Agnostic Gender Debiased Image Captioningcited by 17
  18. 2024New Job, New Gender? Measuring the Social Bias in Image Generation Modelscited by 15
  19. 2025AI Sees Your Location, But With A Bias Toward The Wealthy Worldcited by 14
  20. 2023GD-COMET: A Geo-Diverse Commonsense Inference Modelcited by 13
  21. 2024FairDeDup: Detecting and Mitigating Vision-Language Fairness Disparities in Semantic Dataset Deduplicationcited by 11
  22. 2023Adventures of Trustworthy Vision-Language Models: A Surveycited by 10
  23. 2023DIG In: Evaluating Disparities in Image Generations with Indicators for Geographic Diversitycited by 9
  24. 2022Trustworthy Social Bias Measurementcited by 7
  25. 2023VLSlice: Interactive Vision-and-Language Slice Discoverycited by 7
  26. 2024Resampled Datasets Are Not Enough: Mitigating Societal Bias Beyond Single Attributescited by 7
  27. 2024Image-Perfect Imperfections: Safety, Bias, and Authenticity in the Shadow of Text-To-Image Model Evolutioncited by 7
  28. 2024VersusDebias: Universal Zero-Shot Debiasing for Text-to-Image Models via SLM-Based Prompt Engineering and Generative Adversarycited by 6
  29. 2025ReCon: Region-Controllable Data Augmentation with Rectification and Alignment for Object Detectioncited by 6
  30. 2021Toward Deconfounding the Influence of Entity Demographics for Question Answering Accuracycited by 5
  31. 2025Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluationcited by 5
  32. 2022A survey of Identification and mitigation of Machine Learning algorithmic biases in Image Analysiscited by 3
  33. 2024ML-EAT: A Multilevel Embedding Association Test for Interpretable and Transparent Social Sciencecited by 3
  34. 2025VIGNETTE: Socially Grounded Bias Evaluation for Vision-Language Modelscited by 2
  35. 2025FAME: Fairness-aware Attention-modulated Video Editingcited by 2
  36. 2021Proposing an Interactive Audit Pipeline for Visual Privacy Researchcited by 1
  37. 2025MASS: Overcoming Language Bias in Image-Text Matchingcited by 1
  38. 2022NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries?cited by 0

50 most cited of 61 All 61 (JSON)