mlsanityData to 7 Oct 2026

Your fairness may vary: Pretrained language model fairness in toxic text classification

arXiv 2108.01250 ↗First posted 3 Aug 2021cs.CL cs.LGcited by 38

Not on the map. Why?

Abstract (arXiv)

The popularity of pretrained language models in natural language processing systems calls for a careful evaluation of such models in down-stream tasks, which have a higher potential for societal impact. The evaluation of such systems usually focuses on accuracy measures. Our findings in this paper call for attention to be paid to fairness measures as well. Through the analysis of more than a dozen pretrained language models of varying sizes on two toxic text classification tasks (English), we demonstrate that focusing on accuracy measures alone can lead to models with wide variation in fairness characteristics. Specifically, we observe that fairness can vary even more than accuracy with increasing training data size and different random initializations. At the same time, we find that little of the fairness variation is explained by model size, despite claims in the literature. To improve model fairness without retraining, we show that two post-processing methods developed for structured, tabular data can be successfully applied to a range of pretrained language models. Warning: This paper contains samples of offensive text.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 45 of its 82 references that are in the map. First posted on arXiv 3 Aug 2021.

Builds on

45 of 82 references.

References · 45

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2017On Calibration of Modern Neural Networkscited by 4,860
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  8. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  9. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  10. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  11. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  12. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
33 more
  1. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  2. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  3. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  4. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  5. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  6. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  7. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 476
  8. 2017On Fairness and Calibrationcited by 406
  9. 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 403
  10. 2019Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classificationcited by 396
  11. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  12. 2020BERTweet: A pre-trained language model for English Tweetscited by 346
  13. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  14. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 222
  15. 2018Multiaccuracy: Black-Box Post-Processing for Fairness in Classificationcited by 221
  16. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  17. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 179
  18. 2020Characterising Bias in Compressed Modelscited by 162
  19. 2020Funnel-Transformer: Filtering out Sequential Redundancy for Efficient Language Processingcited by 161
  20. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  21. 2018The Frontiers of Fairness in Machine Learningcited by 139
  22. 2019Wasserstein Fair Classificationcited by 131
  23. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  24. 2016A Confidence-Based Approach for Balancing Fairness and Accuracycited by 73
  25. 2020Mapping the Space of Chemical Reactions Using Attention-Based Neural Networkscited by 70
  26. 2019Inferring Which Medical Treatments Work from Reports of Clinical Trialscited by 63
  27. 2019Leveraging Labeled and Unlabeled Data for Consistent Fair Binary Classificationcited by 60
  28. 2021Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Modelscited by 57
  29. 2020SqueezeBERT: What can computer vision teach NLP about efficient neural networks?cited by 50
  30. 2019Equalized odds postprocessing under imperfect group informationcited by 48
  31. 2020Confronting Abusive Language Online: A Survey from the Ethical and Human Rights Perspectivecited by 43
  32. 2019Fairness in representation: quantifying stereotyping as a representational harmcited by 26
  33. 2019Optimized Score Transformation for Consistent Fair Classificationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 5; 2023: 19; 2024: 10; 2025: 3; 2026: 1.

38 papers.

Sort:

By area:

  1. 2023A Survey on Fairness in Large Language Modelscited by 91
  2. 2023Distilling Script Knowledge from Large Language Models for Constrained Language Planningcited by 45
  3. 2024Threats, Attacks, and Defenses in Machine Unlearning: A Surveycited by 34
  4. 2022The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarkscited by 31
  5. 2023Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectivescited by 30
  6. 2023On The Impact of Machine Learning Randomness on Group Fairnesscited by 26
  7. 2023Grounding Foundation Models through Federated Transfer Learning: A General Frameworkcited by 24
  8. 2023How do different tokenizers perform on downstream tasks in scriptio continua languages?: A case study in Japanesecited by 20
  9. 2022Vote'n'Rank: Revision of Benchmarking with Social Choice Theorycited by 15
  10. 2023SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Modelscited by 14
  11. 2023Being Right for Whose Right Reasons?cited by 12
  12. 2024Rapid Optimization for Jailbreaking LLMs via Subconscious Exploitation and Echopraxiacited by 12
26 more
  1. 2022Conditional Supervised Contrastive Learning for Fair Text Classificationcited by 11
  2. 2023FairSISA: Ensemble Post-Processing to Improve Fairness of Unlearning in LLMscited by 10
  3. 2024Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitationscited by 10
  4. 2023Fairness Evaluation in Text Classification: Machine Learning Practitioner Perspectives of Individual and Group Fairnesscited by 7
  5. 2023An Empirical Investigation into Benchmarking Model Multiplicity for Trustworthy Machine Learning: A Case Study on Image Classificationcited by 7
  6. 2022Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Controlcited by 6
  7. 2023On the Origins of Bias in NLP through the Lens of the Jim Codecited by 5
  8. 2024The Case for Globalizing Fairness: A Mixed Methods Study on Colonialism, AI, and Health in Africacited by 5
  9. 2023Location-Aware Visual Question Generation with Lightweight Modelscited by 4
  10. 2022Systematic Evaluation of Predictive Fairnesscited by 3
  11. 2024Dynamic Normativity: Necessary and Sufficient Conditions for Value Alignmentcited by 3
  12. 2024Inducing Group Fairness in Prompt-Based Language Model Decisionscited by 3
  13. 2024Different Horses for Different Courses: Comparing Bias Mitigation Algorithms in MLcited by 3
  14. 2025HInter: Exposing Hidden Intersectional Bias in Large Language Modelscited by 3
  15. 2025Mitigating Confounding in Speech-Based Dementia Detection through Weight Maskingcited by 3
  16. 2023How does the task complexity of masked pretraining objectives affect downstream performance?cited by 2
  17. 2024The Trade-off between Performance, Efficiency, and Fairness in Adapter Modules for Text Classificationcited by 2
  18. 2023Linking convolutional kernel size to generalization bias in face analysis CNNscited by 1
  19. 2023On Bias and Fairness in NLP: Investigating the Impact of Bias and Debiasing in Language Models on the Fairness of Toxicity Detectioncited by 1
  20. 2023Selecting Shots for Demographic Fairness in Few-Shot Learning with Large Language Modelscited by 1
  21. 2024Alternative Speech: Complementary Method to Counter-Narrative for Better Discoursecited by 1
  22. 2023Globalizing Fairness Attributes in Machine Learning: A Case Study on Health in Africacited by 0
  23. 2023Keeping Up with the Language Models: Systematic Benchmark Extension for Bias Auditingcited by 0
  24. 2024fairBERTs: Erasing Sensitive Information Through Semantic and Fairness-aware Perturbationscited by 0
  25. 2025Group Fairness Meets the Black Box: Enabling Fair Algorithms on Closed LLMs via Post-Processingcited by 0
  26. 2026Fairness Beyond a Single Run: Training-Seed Variability in Speech LLM Adaptationcited by 0