mlsanityData to 14 Oct 2026

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems

arXiv 2506.04482 ↗First posted 4 Jun 2025cs.CY cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

The NLP research community has made publicly available numerous instruments for measuring representational harms caused by large language model (LLM)-based systems. These instruments have taken the form of datasets, metrics, tools, and more. In this paper, we examine the extent to which such instruments meet the needs of practitioners tasked with evaluating LLM-based systems. Via semi-structured interviews with 12 such practitioners, we find that practitioners are often unable to use publicly available instruments for measuring representational harms. We identify two types of challenges. In some cases, instruments are not useful because they do not meaningfully measure what practitioners seek to measure or are otherwise misaligned with practitioner needs. In other cases, instruments - even useful instruments - are not used by practitioners due to practical and institutional barriers impeding their uptake. Drawing on measurement theory and pragmatic measurement, we provide recommendations for addressing these challenges to better meet practitioner needs.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 38 of its 93 references that are in the map. First posted on arXiv 4 Jun 2025.

Builds on

38 of 93 references.

References · 38

  1. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  2. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  3. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  4. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  5. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  6. 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
  7. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  8. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
  9. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  10. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 541
  11. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  12. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
26 more
  1. 2019On Measuring Social Biases in Sentence Encoderscited by 398
  2. 2019Measurement and Fairnesscited by 308
  3. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
  4. 2017Why We Need New Evaluation Metrics for NLGcited by 233
  5. 2020HateBERT: Retraining BERT for Abusive Language Detection in Englishcited by 207
  6. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  7. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 185
  8. 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 155
  9. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  10. 2021Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologiescited by 128
  11. 2021On Measures of Biases and Harms in NLPcited by 81
  12. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  13. 2023You don't need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instrumentscited by 48
  14. 2023Evaluating Evaluation Metrics: A Framework for Analyzing NLG Evaluation Metrics using Measurement Theorycited by 43
  15. 2022Quantifying Social Biases Using Templates is Unreliablecited by 39
  16. 2022Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkitscited by 34
  17. 2022Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implicationscited by 32
  18. 2024Towards AI Accountability Infrastructure: Gaps and Opportunities in AI Audit Toolingcited by 29
  19. 2024Position: Measure Dataset Diversity, Don't Just Claim Itcited by 26
  20. 2023Taxonomizing and Measuring Representational Harms: A Look at Image Taggingcited by 25
  21. 2023''Fifty Shades of Bias'': Normative Ratings of Gender Bias in GPT Generated English Textcited by 15
  22. 2021Assessing the Reliability of Word Embedding Gender Bias Measurescited by 14
  23. 2023The Validity of Evaluation Results: Assessing Concurrence Across Compositionality Benchmarkscited by 12
  24. 2023A Framework for Automated Measurement of Responsible AI Harms in Generative AI Applicationscited by 12
  25. 2022Trustworthy Social Bias Measurementcited by 7
  26. 2023Challenges to Evaluating the Generalization of Coreference Resolution Models: A Measurement Modeling Perspectivecited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigorcited by 7
  2. 2025Measurement as Bricolage: Examining How Data Scientists Construct Target Variables for Predictive Modeling Taskscited by 1
  3. 2026Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Modelscited by 0