mlsanityData to 7 Oct 2026

Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting

arXiv 1901.09451 ↗First posted 27 Jan 2019cs.IR cs.LG stat.MLcited by 346

Not on the map. Why?

Abstract (arXiv)

We present a large-scale study of gender bias in occupation classification, a task where the use of machine learning may lead to negative outcomes on peoples' lives. We analyze the potential allocation harms that can result from semantic representation bias. To do so, we study the impact on occupation classification of including explicit gender indicators---such as first names and pronouns---in different semantic representations of online biographies. Additionally, we quantify the bias that remains when these indicators are "scrubbed," and describe proxy behavior that occurs in the absence of explicit gender indicators. As we demonstrate, differences in true positive rates between genders are correlated with existing gender imbalances in occupations, which may compound these imbalances.

Key facts. Cited by 346 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 42; 2022: 68; 2023: 57; 2024: 70; 2025: 59; 2026: 50.

346 papers.

Sort:

By area:

  1. 2023Whose Opinions Do Language Models Reflect?cited by 759
  2. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  3. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  4. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  5. 2023LEACE: Perfect linear concept erasure in closed formcited by 254
  6. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 196
  7. 2023Knowledge Editing for Large Language Models: A Surveycited by 189
  8. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  9. 2023Inspecting and Editing Knowledge Representations in Language Modelscited by 140
  10. 2021Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologiescited by 128
  11. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  12. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
38 more
  1. 2022Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Datasetcited by 113
  2. 2021Measure and Improve Robustness in NLP Models: A Surveycited by 107
  3. 2023How do Language Models Bind Entities in Context?cited by 102
  4. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
  5. 2022Linear Adversarial Concept Erasurecited by 98
  6. 2023A Survey on Fairness in Large Language Modelscited by 91
  7. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  8. 2023Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMscited by 82
  9. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 82
  10. 2021On Measures of Biases and Harms in NLPcited by 81
  11. 2024Bias in Large Language Models: Origin, Evaluation, and Mitigationcited by 77
  12. 2021Contrastive Explanations for Model Interpretabilitycited by 75
  13. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  14. 2021A Survey of Race, Racism, and Anti-Racism in NLPcited by 74
  15. 2023Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanationscited by 73
  16. 2023The Bias Amplification Paradox in Text-to-Image Generationcited by 69
  17. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  18. 2021Post-processing for Individual Fairnesscited by 61
  19. 2021Understanding the Effect of Out-of-distribution Examples and Interactive Explanations on Human-AI Decision Makingcited by 58
  20. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  21. 2022Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Modelscited by 56
  22. 2021Explanation-Based Human Debugging of NLP Models: A Surveycited by 53
  23. 2022Perturbation Augmentation for Fairer NLPcited by 51
  24. 2022Fair and Optimal Classification via Post-Processingcited by 50
  25. 2021Diverse Adversaries for Mitigating Bias in Trainingcited by 48
  26. 2022Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Taskscited by 48
  27. 2022Kernelized Concept Erasurecited by 47
  28. 2024Representation Surgery: Theory and Practice of Affine Steeringcited by 47
  29. 2024Disclosure and Mitigation of Gender Bias in LLMscited by 47
  30. 2022A Causal Lens for Controllable Text Generationcited by 45
  31. 2021SelfExplain: A Self-Explaining Architecture for Neural Text Classifierscited by 44
  32. 2024Bias in Language Models: Beyond Trick Tests and Toward RUTEd Evaluationcited by 41
  33. 2022Mitigating Gender Bias in Distilled Language Models via Counterfactual Role Reversalcited by 40
  34. 2024Digital Forgetting in Large Language Models: A Survey of Unlearning Methodscited by 38
  35. 2022Representation Bias in Data: A Survey on Identification and Resolution Techniquescited by 37
  36. 2021Societal Biases in Retrieved Contents: Measurement Framework and Adversarial Mitigation for BERT Rankerscited by 34
  37. 2021Balancing out Bias: Achieving Fairness Through Balanced Trainingcited by 34
  38. 2021Contrastive Learning for Fair Representationscited by 34

50 most cited of 346 All 346 (JSON)