mlsanityData to 7 Oct 2026

Semantics derived automatically from language corpora contain human-like biases

arXiv 1608.07187 ↗First posted 25 Aug 2016cs.AI cs.CL cs.CYcited by 894

Not on the map. Why?

Cited by 24 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Artificial intelligence and machine learning are in a period of astounding growth. However, there are concerns that these technologies may be used, either with or without intention, to perpetuate the prejudice and unfairness that unfortunately characterizes many human institutions. Here we show for the first time that human-like semantic biases result from the application of standard machine learning to ordinary language---the same sort of language humans are exposed to every day. We replicate a spectrum of standard human biases as exposed by the Implicit Association Test and other well-known psychological studies. We replicate these using a widely used, purely statistical machine-learning model---namely, the GloVe word embedding---trained on a corpus of text from the Web. Our results indicate that language itself contains recoverable and accurate imprints of our historic biases, whether these are morally neutral as towards insects or flowers, problematic as towards race or gender, or even simply veridical, reflecting the em status quo for the distribution of gender with respect to careers or first names. These regularities are captured by machine learning along with the rest of semantics. In addition to our empirical findings concerning language, we also contribute new methods for evaluating bias in text, the Word Embedding Association Test (WEAT) and the Word Embedding Factual Association Test (WEFAT). Our results have implications not only for AI and machine learning, but also for the fields of psychology, sociology, and human ethics, since they raise the possibility that mere exposure to everyday language can account for the biases we replicate here.

Key facts. Cited by 894 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 130; 2022: 135; 2023: 174; 2024: 196; 2025: 152; 2026: 107.

894 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Ethical and social risks of harm from Language Modelscited by 957
  3. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  4. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 737
  5. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  6. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  7. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  8. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  9. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  10. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 443
  11. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  12. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
38 more
  1. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  2. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  3. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  4. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  5. 2023PaLI-X: On Scaling up a Multilingual Vision and Language Modelcited by 254
  6. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  7. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  8. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  9. 2023Gender bias and stereotypes in Large Language Modelscited by 185
  10. 2023The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientationcited by 183
  11. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  12. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
  13. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  14. 2023A Study of Generative Large Language Model for Medical Research and Healthcarecited by 160
  15. 2023In-Context Impersonation Reveals Large Language Models' Strengths and Biasescited by 154
  16. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
  17. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  18. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  19. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 147
  20. 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 145
  21. 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 144
  22. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 143
  23. 2022CM3: A Causal Masked Multimodal Model of the Internetcited by 142
  24. 2021Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Modelscited by 141
  25. 2023Auditing large language models: a three-layered approachcited by 135
  26. 2023Large language models can accurately predict searcher preferencescited by 129
  27. 2021Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologiescited by 128
  28. 2021What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Researchcited by 119
  29. 2023Generative AIcited by 119
  30. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
  31. 2021Gender Bias in Machine Translationcited by 116
  32. 2021Understanding and Evaluating Racial Biases in Image Captioningcited by 108
  33. 2021Debiasing Pre-trained Contextualised Embeddingscited by 107
  34. 2023Generative Language Models Exhibit Social Identity Biasescited by 107
  35. 2022A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learningcited by 104
  36. 2021Sustainable Modular Debiasing of Language Modelscited by 98
  37. 2022Linear Adversarial Concept Erasurecited by 98
  38. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96

50 most cited of 894 All 894 (JSON)