mlsanityData to 7 Oct 2026

FineDeb: A Debiasing Framework for Language Models

arXiv 2302.02453 ↗First posted 5 Feb 2023cs.CL cs.CYcited by 7

Not on the map. Why?

Abstract (arXiv)

As language models are increasingly included in human-facing machine learning tools, bias against demographic subgroups has gained attention. We propose FineDeb, a two-phase debiasing framework for language models that starts with contextual debiasing of embeddings learned by pretrained language models. The model is then fine-tuned on a language modeling objective. Our results show that FineDeb offers stronger debiasing in comparison to other methods which often result in models as biased as the original language model. Our framework is generalizable for demographics with multiple classes, and we demonstrate its effectiveness through extensive experiments and comparisons with state of the art techniques. We release our code and data on GitHub.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 19 of its 25 references that are in the map. First posted on arXiv 5 Feb 2023.

Builds on

19 of 25 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  3. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  4. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  5. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  6. 2015Teaching Machines to Read and Comprehendcited by 1,039
  7. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  8. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  9. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  10. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  11. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  12. 2019On Measuring Social Biases in Sentence Encoderscited by 397
7 more
  1. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  2. 2020Towards Debiasing Sentence Representationscited by 170
  3. 2018Learning Gender-Neutral Word Embeddingscited by 169
  4. 2021RedditBias: A Real-World Resource for Bias Evaluation and Debiasing of Conversational Language Modelscited by 149
  5. 2019Black is to Criminal as Caucasian is to Police: Detecting and Removing Multiclass Bias in Word Embeddingscited by 146
  6. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 96
  7. 2019Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Functioncited by 57

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 2; 2026: 1.

7 papers.

  1. 2023AutoML in the Age of Large Language Models: Current Challenges, Future Opportunities and Riskscited by 34
  2. 2023OpinionGPT: Modelling Explicit Biases in Instruction-Tuned LLMscited by 29
  3. 2024Ethos: Rectifying Language Models in Orthogonal Parameter Spacecited by 20
  4. 2024From Prejudice to Parity: A New Approach to Debiasing Large Language Model Word Embeddingscited by 3
  5. 2025McBE: A Multi-task Chinese Bias Evaluation Benchmark for Large Language Modelscited by 3
  6. 2025KLAAD: Refining Attention Mechanisms to Reduce Societal Bias in Generative Language Modelscited by 3
  7. 2026RobustDebias: Debiasing Language Models using Distributionally Robust Optimizationcited by 1