mlsanityData to 14 Oct 2026

Fairness in Language Models Beyond English: Gaps and Challenges

arXiv 2302.12578 ↗First posted 24 Feb 2023cs.CL cs.CY cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

With language models becoming increasingly ubiquitous, it has become essential to address their inequitable treatment of diverse demographic groups and factors. Most research on evaluating and mitigating fairness harms has been concentrated on English, while multilingual models and non-English languages have received comparatively little attention. This paper presents a survey of fairness in multilingual and non-English contexts, highlighting the shortcomings of current research and the difficulties faced by methods designed for English. We contend that the multitude of diverse cultures and languages across the world makes it infeasible to achieve comprehensive coverage in terms of constructing fairness datasets. Thus, the measurement and mitigation of biases must evolve beyond the current dataset-driven practices that are narrowly focused on specific dimensions and types of biases and, therefore, impossible to scale across languages and cultures.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 62 of its 117 references that are in the map. First posted on arXiv 24 Feb 2023.

Builds on

62 of 117 references.

References · 62

  1. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  2. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
  3. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  4. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
  5. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  6. 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
  7. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
  8. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  9. 2019On Measuring Social Biases in Sentence Encoderscited by 398
  10. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  11. 2019Measuring Bias in Contextualized Word Representationscited by 276
  12. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 264
38 more
  1. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 224
  2. 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
  3. 2021Visually Grounded Reasoning across Languages and Culturescited by 189
  4. 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 171
  5. 2020Characterising Bias in Compressed Modelscited by 167
  6. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  7. 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 135
  8. 2021Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologiescited by 128
  9. 2021Re-imagining Algorithmic Fairness in India and Beyondcited by 104
  10. 2021Sustainable Modular Debiasing of Language Modelscited by 100
  11. 2021An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Modelscited by 97
  12. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  13. 2017Ethical Challenges in Data-Driven Dialogue Systemscited by 86
  14. 2019Toward Gender-Inclusive Coreference Resolutioncited by 84
  15. 2022Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Surveycited by 80
  16. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  17. 2017Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American Englishcited by 71
  18. 2020Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairnesscited by 67
  19. 2022One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesiacited by 66
  20. 2016Cultural Shift or Linguistic Drift? Comparing Two Computational Measures of Semantic Changecited by 63
  21. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  22. 2021The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writerscited by 61
  23. 2019On Measuring Gender Bias in Translation of Gender-neutral Pronounscited by 60
  24. 2021Unmasking the Mask -- Evaluating Social Biases in Masked Language Modelscited by 60
  25. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  26. 2019Understanding Undesirable Word Embedding Associationscited by 59
  27. 2020Gender Bias in Multilingual Embeddings and Cross-Lingual Transfercited by 55
  28. 2022Re-contextualizing Fairness in NLP: The Case of Indiacited by 55
  29. 2022Welcome to the Modern World of Pronouns: Identity-Inclusive Natural Language Processing beyond Gendercited by 47
  30. 2022Don't Blame the Annotator: Bias Already Starts in the Annotation Instructionscited by 47
  31. 2020Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognitioncited by 43
  32. 2022Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Biascited by 43
  33. 2019Examining Gender Bias in Languages with Grammatical Gendercited by 40
  34. 2022American == White in Multimodal Language-and-Image AIcited by 39
  35. 2015Computational Sociolinguistics: A Surveycited by 35
  36. 2021Assessing Multilingual Fairness in Pre-trained Multimodal Representationscited by 35
  37. 2022Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Biascited by 33
  38. 2021Evaluating Debiasing Techniques for Intersectional Biasescited by 30

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 6; 2025: 2; 2026: 4.

16 papers.

Sort:

By area:

  1. 2023MEGA: Multilingual Evaluation of Generative AIcited by 293
  2. 2023Language Model Tokenizers Introduce Unfairness Between Languagescited by 214
  3. 2024Towards Conversational Diagnostic AIcited by 79
  4. 2023On the application of Large Language Models for language teaching and assessment technologycited by 27
  5. 2024BertaQA: How Much Do Language Models Know About Local Culture?cited by 23
  6. 2023BHASA: A Holistic Southeast Asian Linguistic and Cultural Evaluation Suite for Large Language Modelscited by 18
  7. 2024The Sociolinguistic Foundations of Language Modelingcited by 13
  8. 2025Lost in Cultural Translation: Do LLMs Struggle with Math Across Cultural Contexts?cited by 9
  9. 2024LLeMpower: Understanding Disparities in the Control and Access of Large Language Modelscited by 5
  10. 2024Standard Language Ideology in AI-Generated Languagecited by 5
  11. 2024Distributed agency in second language learning and teaching through generative AIcited by 4
  12. 2025Do Chinese models speak Chinese languages?cited by 2
4 more
  1. 2026Multilingual Large Language Models do not comprehend all natural languages to equal degreescited by 1
  2. 2026Red Teaming LLMs as Socio-Technical Practice: From Exploration and Data Creation to Evaluationcited by 0
  3. 2026Multiple-Debias: A Full-process Debiasing Method for Multilingual Pre-trained Language Modelscited by 0
  4. 2026Per-Entity Bias Mapping for AI Visibility: Why Brand Mentions Require Entity-Specific Calibrationcited by 0