mlsanityData to 7 Oct 2026

Challenges in Measuring Bias via Open-Ended Language Generation

arXiv 2205.11601 ↗First posted 23 May 2022cs.CL cs.CYcited by 23

Not on the map. Why?

Abstract (arXiv)

Researchers have devised numerous ways to quantify social biases vested in pretrained language models. As some language models are capable of generating coherent completions given a set of textual prompts, several prompting datasets have been proposed to measure biases between social groups -- posing language generation as a way of identifying biases. In this opinion paper, we analyze how specific choices of prompt sets, metrics, automatic tools and sampling strategies affect bias results. We find out that the practice of measuring biases through text completion is prone to yielding contradicting results under different experiment settings. We additionally provide recommendations for reporting biases in open-ended language generation for a more complete outlook of biases exhibited by a given language model. Code to reproduce the results is released under https://github.com/feyzaakyurek/bias-textgen.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 16 of its 28 references that are in the map. First posted on arXiv 23 May 2022.

Builds on

16 of 28 references.

References · 16

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  4. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  5. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  6. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  7. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  8. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  9. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  10. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 184
  11. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  12. 2021Challenges in Detoxifying Language Modelscited by 169
4 more
  1. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  2. 2021They, Them, Theirs: Rewriting with Gender-Neutral Englishcited by 44
  3. 2021Bidimensional Leaderboards: Generate and Evaluate Language Hand in Handcited by 32
  4. 2017Neural-based Natural Language Generation in Dialogue using RNN Encoder-Decoder with Semantic Aggregationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 7; 2024: 4; 2025: 5; 2026: 3.

23 papers.

Sort:

  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  2. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  3. 2023Evaluation of African American Language Bias in Natural Language Generationcited by 36
  4. 2024Decoding Biases: Automated Methods and LLM Judges for Gender Bias Detection in Language Modelscited by 33
  5. 2022Unified Detoxifying and Debiasing in Language Generation via Inference-time Adaptive Optimizationcited by 32
  6. 2022Causally Testing Gender Bias in LLMs: A Case Study on Occupational Biascited by 28
  7. 2024ECBD: Evidence-Centered Benchmark Design for NLPcited by 27
  8. 2023DUnE: Dataset for Unified Editingcited by 19
  9. 2023Tackling Bias in Pre-trained Language Models: Current Trends and Under-represented Societiescited by 11
  10. 2023This Prompt is Measuring <MASK>: Evaluating Bias Evaluation in Language Modelscited by 9
  11. 2023In-Depth Look at Word Filling Societal Bias Measurescited by 8
  12. 2022An Analysis of the Effects of Decoding Algorithms on Fairness in Open-Ended Language Generationcited by 7
11 more
  1. 2024KidLM: Advancing Language Models for Children -- Early Insights and Future Directionscited by 7
  2. 2025Agree to Disagree? A Meta-Evaluation of LLM Misgenderingcited by 6
  3. 2022Undesirable Biases in NLP: Addressing Challenges of Measurementcited by 4
  4. 2025Sports and Women's Sports: Gender Bias in Text Generation with Olympic Datacited by 3
  5. 2025Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Languagecited by 3
  6. 2026When LLMs Imagine People: A Human-Centered Persona Brainstorm Audit for Bias and Fairness in Creative Applicationscited by 2
  7. 2025GG-BBQ: German Gender Bias Benchmark for Question Answeringcited by 1
  8. 2026AI Evaluation Should Require Standardized Item-Level Data Releasescited by 1
  9. 2024Towards Resource Efficient and Interpretable Bias Mitigation in Large Language Modelscited by 0
  10. 2025Developing A Framework to Support Human Evaluation of Bias in Generated Free Response Textcited by 0
  11. 2026GPF-LiveNews: A Streaming Evaluation Protocol for Group-Conditioned Framing in Large Language Modelscited by 0