mlsanityData to 14 Oct 2026

"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Dataset

arXiv 2205.09209 ↗First posted 18 May 2022cs.CL cs.CYcited by 162

Not on the map. Why?

Abstract (arXiv)

As language models grow in popularity, it becomes increasingly important to clearly measure all possible markers of demographic identity in order to avoid perpetuating existing societal harms. Many datasets for measuring bias currently exist, but they are restricted in their coverage of demographic axes and are commonly used with preset bias tests that presuppose which types of biases models can exhibit. In this work, we present a new, more inclusive bias measurement dataset, HolisticBias, which includes nearly 600 descriptor terms across 13 different demographic axes. HolisticBias was assembled in a participatory process including experts and community members with lived experience of these terms. These descriptors combine with a set of bias measurement templates to produce over 450,000 unique sentence prompts, which we use to explore, identify, and reduce novel forms of bias in several generative models. We demonstrate that HolisticBias is effective at measuring previously undetectable biases in token likelihoods from language models, as well as in an offensiveness classifier. We will invite additions and amendments to the dataset, which we hope will serve as a basis for more easy-to-use and standardized methods for evaluating bias in NLP models.

Key facts. Cited by 162 papers in the map (data through 14 Oct 2026). Builds on 52 of its 89 references that are in the map. First posted on arXiv 18 May 2022.

Builds on

52 of 89 references.

References · 52

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  3. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  4. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  5. 2022Red Teaming Language Models with Language Modelscited by 1,026
  6. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  7. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  8. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  9. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  10. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  11. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
  12. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
38 more
  1. 2020Recipes for building an open-domain chatbotcited by 593
  2. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  3. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  4. 2020Towards a Human-like Open-Domain Chatbotcited by 470
  5. 2021Dynabench: Rethinking Benchmarking in NLPcited by 459
  6. 2018Gender Bias in Coreference Resolutioncited by 442
  7. 2019On Measuring Social Biases in Sentence Encoderscited by 398
  8. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
  9. 2020The Pushshift Reddit Datasetcited by 292
  10. 2020Evaluation of Text Generation: A Surveycited by 277
  11. 2021Beyond Goldfish Memory: Long-Term Open-Domain Conversationcited by 267
  12. 2021The GEM Benchmark: Natural Language Generation, its Evaluation and Metricscited by 239
  13. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 224
  14. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  15. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  16. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  17. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 185
  18. 2019The Second Conversational Intelligence Challenge (ConvAI2)cited by 179
  19. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  20. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  21. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  22. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  23. 2017ParlAI: A Dialog Research Software Platformcited by 139
  24. 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 135
  25. 2020Can You Put it All Together: Evaluating Conversational Agents' Ability to Blend Skillscited by 131
  26. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  27. 2018Retrieve and Refine: Improved Sequence Generation Models For Dialoguecited by 82
  28. 2020Multi-Dimensional Gender Bias Classificationcited by 80
  29. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  30. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  31. 2021Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contextscited by 56
  32. 2020Controlling Style in Generated Dialoguecited by 38
  33. 2021Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compressioncited by 38
  34. 2022AugLy: Data Augmentations for Robustnesscited by 34
  35. 2021Revealing Persona Biases in Dialogue Systemscited by 33
  36. 2021Gender Bias Amplification During Speed-Quality Optimization in Neural Machine Translationcited by 28
  37. 2021What Ingredients Make for an Effective Crowdsourcing Protocol for Difficult NLU Data Collection Tasks?cited by 23
  38. 2021Hi, my name is Martha: Using names to measure and mitigate bias in generative dialogue modelscited by 23

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 41; 2024: 63; 2025: 24; 2026: 32.

162 papers.

Sort:

By area:

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  3. 2023PaLM 2 Technical Reportcited by 1,162
  4. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  5. 2023Large Language Model Alignment: A Surveycited by 243
  6. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
  7. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 207
  8. 2023Gender bias and stereotypes in Large Language Modelscited by 187
  9. 2023Having Beer after Prayer? Measuring Cultural Bias in Large Language Modelscited by 153
  10. 2024Spirit LM: Interleaved Spoken and Written Language Modelcited by 126
  11. 2023A Survey on Fairness in Large Language Modelscited by 95
  12. 2024Large Language Models are Geographically Biasedcited by 93
38 more
  1. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  2. 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
  3. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 77
  4. 2024The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harmcited by 70
  5. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  6. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 65
  7. 2023TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Modelscited by 58
  8. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 58
  9. 2023SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examplescited by 53
  10. 2023Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Taskscited by 51
  11. 2023Supporting Human-AI Collaboration in Auditing LLMs with LLMscited by 46
  12. 2023Improving Diversity of Demographic Representation in Large Language Models via Collective-Critiques and Self-Votingcited by 46
  13. 2023WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Modelscited by 42
  14. 2024Social Bias Evaluation for Large Language Models Requires Prompt Variationscited by 40
  15. 2023Are Personalized Stochastic Parrots More Dangerous? Evaluating Persona Biases in Dialogue Systemscited by 37
  16. 2024Emulated Disalignment: Safety Alignment for Large Language Models May Backfire!cited by 37
  17. 2024FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMscited by 36
  18. 2024MBBQ: A Dataset for Cross-Lingual Comparison of Stereotypes in Generative LLMscited by 35
  19. 2024Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compressioncited by 33
  20. 2024Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Modelscited by 33
  21. 2023Investigating Subtler Biases in LLMs: Ageism, Beauty, Institutional, and Nationality Bias in Generative Modelscited by 32
  22. 2023Social Bias Probing: Fairness Benchmarking for Language Modelscited by 32
  23. 2023Vision-Language Models Performing Zero-Shot Tasks Exhibit Gender-based Disparitiescited by 31
  24. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 28
  25. 2024Uncovering Bias in Large Vision-Language Models at Scale with Counterfactualscited by 28
  26. 2025Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-Reflectioncited by 27
  27. 2023FFT: Towards Harmlessness Evaluation and Analysis for LLMs with Factuality, Fairness, Toxicitycited by 25
  28. 2024Holistic Automated Red Teaming for Large Language Models through Top-Down Test Case Generation and Multi-turn Interactioncited by 25
  29. 2023Multilingual Holistic Bias: Extending Descriptors and Patterns to Unveil Demographic Biases in Languages at Scalecited by 24
  30. 2024Can LLMs Recognize Toxicity? A Structured Investigation Framework and Toxicity Metriccited by 23
  31. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  32. 2023Fairness-Aware Structured Pruning in Transformerscited by 22
  33. 2024Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generationcited by 20
  34. 2024PolygloToxicityPrompts: Multilingual Evaluation of Neural Toxic Degeneration in Large Language Modelscited by 20
  35. 2024Fairness and Bias in Multimodal AI: A Surveycited by 20
  36. 2024Breaking Bias, Building Bridges: Evaluation and Mitigation of Social Biases in LLMs via Contact Hypothesiscited by 20
  37. 2024PaCE: Parsimonious Concept Engineering for Large Language Modelscited by 19
  38. 2023Indian-BhED: A Dataset for Measuring India-Centric Biases in Large Language Modelscited by 18

50 most cited of 162 All 162 (JSON)