mlsanityData to 14 Oct 2026

Gender Bias in Transformer Models: A comprehensive survey

arXiv 2306.10530 ↗First posted 18 Jun 2023cs.CL cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Gender bias in artificial intelligence (AI) has emerged as a pressing concern with profound implications for individuals' lives. This paper presents a comprehensive survey that explores gender bias in Transformer models from a linguistic perspective. While the existence of gender bias in language models has been acknowledged in previous studies, there remains a lack of consensus on how to effectively measure and evaluate this bias. Our survey critically examines the existing literature on gender bias in Transformers, shedding light on the diverse methodologies and metrics employed to assess bias. Several limitations in current approaches to measuring gender bias in Transformers are identified, encompassing the utilization of incomplete or flawed metrics, inadequate dataset sizes, and a dearth of standardization in evaluation methods. Furthermore, our survey delves into the potential ramifications of gender bias in Transformers for downstream applications, including dialogue systems and machine translation. We underscore the importance of fostering equity and fairness in these systems by emphasizing the need for heightened awareness and accountability in developing and deploying language technologies. This paper serves as a comprehensive overview of gender bias in Transformer models, providing novel insights and offering valuable directions for future research in this critical domain.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 40 of its 65 references that are in the map. First posted on arXiv 18 Jun 2023.

Builds on

40 of 65 references.

References · 40

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,181
  6. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  7. 2019SciBERT: A Pretrained Language Model for Scientific Textcited by 1,414
  8. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  9. 2021Transformers in Vision: A Surveycited by 1,013
  10. 2021Pre-Trained Models: Past, Present and Futurecited by 319
  11. 2019Evaluating Gender Bias in Machine Translationcited by 249
  12. 2018Women also Snowboard: Overcoming Bias in Captioning Modelscited by 236
28 more
  1. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  2. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  3. 2018Gender Bias in Neural Natural Language Processingcited by 211
  4. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  5. 2023A Categorical Archive of ChatGPT Failurescited by 168
  6. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  7. 2018Assessing Gender Bias in Machine Translation -- A Case Study with Google Translatecited by 121
  8. 2021Gender Bias in Machine Translationcited by 117
  9. 2019Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systemscited by 110
  10. 2018Understanding the Origins of Bias in Word Embeddingscited by 91
  11. 2020Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Biascited by 82
  12. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  13. 2021Stereotype and Skew: Quantifying Gender Bias in Pre-trained and Fine-tuned Language Modelscited by 57
  14. 2020Mitigating Gender Bias in Captioning Systemscited by 50
  15. 2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
  16. 2019Equalized odds postprocessing under imperfect group informationcited by 48
  17. 2020Fairness Metrics: A Comparative Analysiscited by 45
  18. 2020Investigating Gender Bias in BERTcited by 40
  19. 2021Low Frequency Names Exhibit Bias and Overfitting in Contextualizing Language Modelscited by 39
  20. 2022Quantifying Societal Bias Amplification in Image Captioningcited by 37
  21. 2021BiasFinder: Metamorphic Test Generation to Uncover Bias for Sentiment Analysis Systemscited by 24
  22. 2022Gender Bias in Word Embeddings: A Comprehensive Analysis of Frequency, Syntax, and Semanticscited by 24
  23. 2022Gender Biases and Where to Find Them: Exploring Gender Bias in Pre-Trained Transformer-based Language Models Using Movement Pruningcited by 21
  24. 2015The GAP Benchmark Suitecited by 14
  25. 2019What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiringcited by 14
  26. 2023Linguistic ambiguity analysis in ChatGPTcited by 14
  27. 2021Detecting Gender Bias in Transformer-based Models: A Case Study on BERTcited by 8
  28. 2021Assessing gender bias in medical and scientific masked language models with StereoSetcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 0; 2025: 0; 2026: 1.

2 papers.

  1. 2023Unveiling Gender Bias in Terms of Profession Across LLMs: Analyzing and Addressing Sociological Implicationscited by 29
  2. 2026In Transformer We Trust? A Perspective on Transformer Architecture Failure Modescited by 1