mlsanityData to 14 Oct 2026

BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation

arXiv 2101.11718 ↗First posted 27 Jan 2021cs.CL cs.AI cs.LGcited by 412

Not on the map. Why?

Cited by 33 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advances in deep learning techniques have enabled machines to generate cohesive open-ended text when prompted with a sequence of words as context. While these models now empower many downstream applications from conversation bots to automatic storytelling, they have been shown to generate texts that exhibit social biases. To systematically study and benchmark social biases in open-ended language generation, we introduce the Bias in Open-Ended Language Generation Dataset (BOLD), a large-scale dataset that consists of 23,679 English text generation prompts for bias benchmarking across five domains: profession, gender, race, religion, and political ideology. We also propose new automated metrics for toxicity, psycholinguistic norms, and text gender polarity to measure social biases in open-ended text generation from multiple angles. An examination of text generated from three popular language models reveals that the majority of these models exhibit a larger social bias than human-written Wikipedia text across all domains. With these results we highlight the need to benchmark biases in open-ended language generation and caution users of language generation models on downstream tasks to be cognizant of these embedded prejudices.

Key facts. Cited by 412 papers in the map (data through 14 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 27 Jan 2021.

Builds on

30 of 47 references.

References · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  3. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,790
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  5. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  6. 2020Big Bird: Transformers for Longer Sequencescited by 1,684
  7. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  8. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  9. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  10. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  11. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  12. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
18 more
  1. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  2. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  3. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  4. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
  5. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  6. 2018Gender Bias in Coreference Resolutioncited by 442
  7. 2019How to Fine-Tune BERT for Text Classification?cited by 411
  8. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  9. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  10. 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 234
  11. 2019WinoGrande: An Adversarial Winograd Schema Challenge at Scalecited by 204
  12. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 185
  13. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  14. 2020Towards Controllable Biases in Language Generationcited by 112
  15. 2019Fine-grained Sentiment Classification using BERTcited by 38
  16. 2019PaperRobot: Incremental Draft Generation of Scientific Ideascited by 37
  17. 2015It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopediacited by 30
  18. 2020Learning and Evaluating Emotion Lexicons for 91 Languagescited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 30; 2023: 69; 2024: 123; 2025: 94; 2026: 84.

412 papers.

Sort:

By area:

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  4. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  5. 2024Mixtral of Expertscited by 1,823
  6. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  7. 2022Red Teaming Language Models with Language Modelscited by 1,026
  8. 2023Whose Opinions Do Language Models Reflect?cited by 773
  9. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
  10. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  11. 2023Effective Long-Context Scaling of Foundation Modelscited by 303
  12. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
38 more
  1. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 274
  2. 2023Can Large Language Models Transform Computational Social Science?cited by 250
  3. 2023Large Language Model Alignment: A Surveycited by 243
  4. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
  5. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 224
  6. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  7. 2023"Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letterscited by 199
  8. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 184
  9. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 181
  10. 2021Challenges in Detoxifying Language Modelscited by 169
  11. 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 155
  12. 2024Benchmarking LLMs via Uncertainty Quantificationcited by 126
  13. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  14. 2024BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practicescited by 114
  15. 2022The Ghost in the Machine has an American accent: value conflict in GPT-3cited by 105
  16. 2023LLM360: Towards Fully Transparent Open-Source LLMscited by 104
  17. 2024Large Language Models Meet NLP: A Surveycited by 103
  18. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 98
  19. 2023A Survey on Fairness in Large Language Modelscited by 95
  20. 2024Large Language Models are Geographically Biasedcited by 93
  21. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  22. 2023Red teaming ChatGPT via Jailbreaking: Bias, Robustness, Reliability and Toxicitycited by 89
  23. 2024Measuring Political Bias in Large Language Models: What Is Said and How It Is Saidcited by 89
  24. 2021On Measures of Biases and Harms in NLPcited by 81
  25. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 77
  26. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  27. 2021On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmarkcited by 73
  28. 2023KoBBQ: Korean Bias Benchmark for Question Answeringcited by 71
  29. 2022Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Modelscited by 68
  30. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 66
  31. 2022Data Feedback Loops: Model-driven Amplification of Dataset Biasescited by 60
  32. 2024Large Language Model Safety: A Holistic Surveycited by 59
  33. 2023TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Modelscited by 58
  34. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 58
  35. 2022AlexaTM 20B: Few-Shot Learning Using a Large-Scale Multilingual Seq2Seq Modelcited by 57
  36. 2023"I'm fully who I am": Towards Centering Transgender and Non-Binary Voices to Measure Biases in Open Language Generationcited by 56
  37. 2023BiasAsker: Measuring the Bias in Conversational AI Systemcited by 53
  38. 2023All Languages Matter: On the Multilingual Safety of Large Language Modelscited by 53

50 most cited of 412 All 412 (JSON)