mlsanityData to 14 Oct 2026

Why Don't Prompt-Based Fairness Metrics Correlate?

arXiv 2406.05918 ↗First posted 9 Jun 2024cs.CL cs.AI cs.CYcited by 4

Not on the map. Why?

Abstract (arXiv)

The widespread use of large language models has brought up essential questions about the potential biases these models might learn. This led to the development of several metrics aimed at evaluating and mitigating these biases. In this paper, we first demonstrate that prompt-based fairness metrics exhibit poor agreement, as measured by correlation, raising important questions about the reliability of fairness assessment using prompts. Then, we outline six relevant reasons why such a low correlation is observed across existing metrics. Based on these insights, we propose a method called Correlated Fairness Output (CAIRO) to enhance the correlation between fairness metrics. CAIRO augments the original prompts of a given fairness metric by using several pre-trained language models and then selects the combination of the augmented prompts that achieves the highest correlation across metrics. We show a significant improvement in Pearson correlation from 0.3 and 0.18 to 0.90 and 0.98 across metrics for gender and religion biases, respectively. Our code is available at https://github.com/chandar-lab/CAIRO.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 39 of its 54 references that are in the map. First posted on arXiv 9 Jun 2024.

Builds on

39 of 54 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2019Language Models as Knowledge Bases?cited by 1,936
  6. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  8. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  9. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  10. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  11. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  12. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
27 more
  1. 2019On Measuring Social Biases in Sentence Encoderscited by 398
  2. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 364
  3. 2019Measuring Bias in Contextualized Word Representationscited by 276
  4. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 224
  5. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  6. 2018Gender Bias in Neural Natural Language Processingcited by 211
  7. 2019A Unified MRC Framework for Named Entity Recognitioncited by 199
  8. 2019Dice Loss for Data-imbalanced NLP Taskscited by 182
  9. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 162
  10. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  11. 2020Named Entity Recognition as Dependency Parsingcited by 151
  12. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  13. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  14. 2021Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Basescited by 107
  15. 2019E-BERT: Efficient-Yet-Effective Entity Embeddings for BERTcited by 96
  16. 2020Hate Speech Detection and Racial Bias Mitigation in Social Media based on BERT modelcited by 87
  17. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  18. 2022Entropy-based Attention Regularization Frees Unintended Bias Mitigation from Listscited by 55
  19. 2023Bias Against 93 Stigmatized Groups in Masked Language Models and Downstream Sentiment Classification Taskscited by 51
  20. 2020Fast and Accurate Neural CRF Constituency Parsingcited by 50
  21. 2022Debiasing isn't enough! -- On the Effectiveness of Debiasing MLMs and their Social Biases in Downstream Taskscited by 48
  22. 2022Can Prompt Probe Pretrained Language Models? Understanding the Invisible Risks from a Causal Viewcited by 31
  23. 2023On the Independence of Association Bias and Empirical Fairness in Language Modelscited by 29
  24. 2022BLIND: Bias Removal With No Demographicscited by 22
  25. 2023Fairness-Aware Structured Pruning in Transformerscited by 22
  26. 2022Deep Learning on a Healthy Data Diet: Finding Important Examples for Fairnesscited by 21
  27. 2023Learning to Generate Equitable Text in Dialogue from Biased Training Datacited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2024Cross-Lingual Transfer of Debiasing and Detoxification in Multilingual LLMs: An Extensive Investigationcited by 5
  2. 2025Say It Another Way: Auditing LLMs with a User-Grounded Automated Paraphrasing Frameworkcited by 3
  3. 2026When LLMs Imagine People: A Human-Centered Persona Brainstorm Audit for Bias and Fairness in Creative Applicationscited by 2
  4. 2025Theories of "Sexuality" in Natural Language Processing Bias Researchcited by 1