mlsanityData to 7 Oct 2026

Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

arXiv 1903.04561 ↗First posted 11 Mar 2019cs.LG cs.CL stat.MLcited by 396

Not on the map. Why?

Cited by 22 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Unintended bias in Machine Learning can manifest as systemic differences in performance for different demographic groups, potentially compounding existing challenges to fairness in society at large. In this paper, we introduce a suite of threshold-agnostic metrics that provide a nuanced view of this unintended bias, by considering the various ways that a classifier's score distribution can vary across designated groups. We also introduce a large new test set of online comments with crowd-sourced annotations for identity references. We use this to show how our metrics can be used to find new and potentially subtle unintended bias in existing public models.

Key facts. Cited by 396 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 52; 2022: 74; 2023: 85; 2024: 63; 2025: 65; 2026: 57.

396 papers.

Sort:

By area:

  1. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  2. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  3. 2023PaLM 2 Technical Reportcited by 1,154
  4. 2022Editing Models with Task Arithmeticcited by 1,076
  5. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  6. 2023A Comprehensive Overview of Large Language Modelscited by 639
  7. 2021Just Train Twice: Improving Group Robustness without Training Group Informationcited by 522
  8. 2021Towards Out-Of-Distribution Generalization: A Surveycited by 431
  9. 2023Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyondcited by 409
  10. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  11. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  12. 2022UL2: Unifying Language Learning Paradigmscited by 292
38 more
  1. 2023From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Modelscited by 257
  2. 2024MUSE: Machine Unlearning Six-Way Evaluation for Language Modelscited by 251
  3. 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 247
  4. 2023Pretraining Language Models with Human Preferencescited by 245
  5. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 242
  6. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  7. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 187
  8. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 181
  9. 2021Simple data balancing achieves competitive worst-group-accuracycited by 177
  10. 2022Correct-N-Contrast: A Contrastive Approach for Improving Robustness to Spurious Correlationscited by 171
  11. 2021Challenges in Detoxifying Language Modelscited by 169
  12. 2021ExT5: Towards Extreme Multi-Task Scaling for Transfer Learningcited by 161
  13. 2021Charformer: Fast Character Transformers via Gradient-based Subword Tokenizationcited by 158
  14. 2023Composing Parameter-Efficient Modules with Arithmetic Operationscited by 155
  15. 2023FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learningcited by 146
  16. 2022On Feature Learning in the Presence of Spurious Correlationscited by 145
  17. 2022Leveraging Unlabeled Data to Predict Out-of-Distribution Performancecited by 141
  18. 2021On Calibration and Out-of-domain Generalizationcited by 134
  19. 2023Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensionscited by 111
  20. 2021Scale Efficiently: Insights from Pre-training and Fine-tuning Transformerscited by 105
  21. 2022Plex: Towards Reliability using Pretrained Large Model Extensionscited by 102
  22. 2022AUC Maximization in the Era of Big Data and AI: A Surveycited by 100
  23. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 97
  24. 2023Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluationscited by 96
  25. 2021Trustworthy AI: A Computational Perspectivecited by 93
  26. 2022Spread Spurious Attribute: Improving Worst-group Accuracy with Spurious Attribute Estimationcited by 88
  27. 2021Extending the WILDS Benchmark for Unsupervised Adaptationcited by 85
  28. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  29. 2021Leveraging Sparse Linear Layers for Debuggable Deep Networkscited by 82
  30. 2021On Measures of Biases and Harms in NLPcited by 81
  31. 2021Types of Out-of-Distribution Texts and How to Detect Themcited by 80
  32. 2023Model Merging by Uncertainty-Based Gradient Matchingcited by 77
  33. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  34. 2023Efficient Benchmarking of Language Modelscited by 71
  35. 2023Simple and Fast Group Robustness by Automatic Feature Reweightingcited by 68
  36. 2022Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotationcited by 67
  37. 2022Probable Domain Generalization via Quantile Risk Minimizationcited by 66
  38. 2022Contrastive Adapters for Foundation Model Group Robustnesscited by 62

50 most cited of 396 All 396 (JSON)