mlsanityData to 7 Oct 2026

The Woman Worked as a Babysitter: On Biases in Language Generation

arXiv 1909.01326 ↗First posted 3 Sep 2019cs.CL cs.AIcited by 540

Not on the map. Why?

Cited by 30 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a systematic study of biases in natural language generation (NLG) by analyzing text generated from prompts that contain mentions of different demographic groups. In this work, we introduce the notion of the regard towards a demographic, use the varying levels of regard towards different demographics as a defining metric for bias in NLG, and analyze the extent to which sentiment scores are a relevant proxy metric for regard. To this end, we collect strategically-generated text from language models and manually annotate the text with both sentiment and regard scores. Additionally, we build an automatic regard classifier through transfer learning, so that we can analyze biases in unseen text. Together, these methods reveal the extent of the biased nature of language model generations. Our analysis provides a study of biases in NLG, bias metrics and correlated human judgments, and empirical evidence on the usefulness of our annotated dataset.

Key facts. Cited by 540 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Sep 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 59; 2022: 92; 2023: 111; 2024: 130; 2025: 89; 2026: 59.

540 papers.

Sort:

By area:

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  3. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  4. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  5. 2022Red Teaming Language Models with Language Modelscited by 983
  6. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  7. 2022Galactica: A Large Language Model for Sciencecited by 736
  8. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  9. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  10. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  11. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  12. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
38 more
  1. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  2. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 424
  3. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  4. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  5. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  6. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  7. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  8. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  9. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  10. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 284
  11. 2024Is DPO Superior to PPO for LLM Alignment? A Comprehensive Studycited by 262
  12. 2021Societal Biases in Language Generation: Progress and Challengescited by 201
  13. 2022WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creationcited by 200
  14. 2021How Many Data Points is a Prompt Worth?cited by 197
  15. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 196
  16. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  17. 2023Gender bias and stereotypes in Large Language Modelscited by 185
  18. 2023The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientationcited by 183
  19. 2024A Mechanistic Understanding of Alignment Algorithms: A Case Study on DPO and Toxicitycited by 179
  20. 2021Efficient Large Scale Language Modeling with Mixtures of Expertscited by 175
  21. 2021Challenges in Detoxifying Language Modelscited by 169
  22. 2021Alignment of Language Agentscited by 167
  23. 2021Pretrained Transformers as Universal Computation Enginescited by 162
  24. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  25. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  26. 2024Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attackscited by 150
  27. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  28. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  29. 2021Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Modelscited by 141
  30. 2021Can Machines Learn Morality? The Delphi Experimentcited by 140
  31. 2024Investigating Cultural Alignment of Large Language Modelscited by 140
  32. 2023Auditing large language models: a three-layered approachcited by 135
  33. 2024Tamper-Resistant Safeguards for Open-Weight LLMscited by 132
  34. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 130
  35. 2024Jailbreaking Attack against Multimodal Large Language Modelcited by 128
  36. 2023We're Afraid Language Models Aren't Modeling Ambiguitycited by 124
  37. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 118
  38. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116

50 most cited of 540 All 540 (JSON)