mlsanityData to 7 Oct 2026

Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings

arXiv 1607.06520 ↗First posted 21 Jul 2016cs.CL cs.AI cs.LGcited by 1,304

Not on the map. Why?

Cited by 29 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The blind application of machine learning runs the risk of amplifying biases present in data. Such a danger is facing us with word embedding, a popular framework to represent text data as vectors which has been used in many machine learning and natural language processing tasks. We show that even word embeddings trained on Google News articles exhibit female/male gender stereotypes to a disturbing extent. This raises concerns because their widespread use, as we describe, often tends to amplify these biases. Geometrically, gender bias is first shown to be captured by a direction in the word embedding. Second, gender neutral words are shown to be linearly separable from gender definition words in the word embedding. Using these properties, we provide a methodology for modifying an embedding to remove gender stereotypes, such as the association between between the words receptionist and female, while maintaining desired associations such as between the words queen and female. We define metrics to quantify both direct and indirect gender biases in embeddings, and develop algorithms to "debias" the embedding. Using crowd-worker evaluation as well as standard benchmarks, we empirically demonstrate that our algorithms significantly reduce gender bias in embeddings while preserving the its useful properties such as the ability to cluster related concepts and to solve analogy tasks. The resulting embeddings can be used in applications without amplifying gender bias.

Key facts. Cited by 1,304 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Jul 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 203; 2022: 210; 2023: 238; 2024: 265; 2025: 230; 2026: 158.

1,304 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  3. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  4. 2022Red Teaming Language Models with Language Modelscited by 983
  5. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  6. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  7. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  8. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  9. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  10. 2022Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studiescited by 506
  11. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  12. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 443
38 more
  1. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  2. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  3. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  4. 2022Text and Code Embeddings by Contrastive Pre-Trainingcited by 358
  5. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  6. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  7. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 276
  8. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  9. 2023LEACE: Perfect linear concept erasure in closed formcited by 254
  10. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  11. 2022Challenges and Strategies in Cross-Cultural NLPcited by 216
  12. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 207
  13. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  14. 2021What Will it Take to Fix Benchmarking in Natural Language Understanding?cited by 200
  15. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  16. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 196
  17. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  18. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  19. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 191
  20. 2023Gender bias and stereotypes in Large Language Modelscited by 185
  21. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  22. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  23. 2024Towards Measuring and Modeling "Culture" in LLMs: A Surveycited by 166
  24. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  25. 2021Post-hoc Interpretability for Neural NLP: A Surveycited by 155
  26. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
  27. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  28. 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
  29. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  30. 2023Debiasing Vision-Language Models via Biased Promptscited by 147
  31. 2023Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Modelscited by 144
  32. 2021MultiBench: Multiscale Benchmarks for Multimodal Representation Learningcited by 143
  33. 2021Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Modelscited by 141
  34. 2023Large language models can accurately predict searcher preferencescited by 129
  35. 2021Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collectionscited by 128
  36. 2021Evaluating CLIP: Towards Characterization of Broader Capabilities and Downstream Implicationscited by 128
  37. 2021Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologiescited by 128
  38. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 128

50 most cited of 1,304 All 1,304 (JSON)