mlsanityData to 7 Oct 2026

Fairness in Large Language Models: A Taxonomic Survey

arXiv 2404.01349 ↗First posted 31 Mar 2024cs.CL cs.AIcited by 57

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable success across various domains. However, despite their promising performance in numerous real-world applications, most of these algorithms lack fairness considerations. Consequently, they may lead to discriminatory outcomes against certain communities, particularly marginalized populations, prompting extensive study in fair LLMs. On the other hand, fairness in LLMs, in contrast to fairness in traditional machine learning, entails exclusive backgrounds, taxonomies, and fulfillment techniques. To this end, this survey presents a comprehensive overview of recent advances in the existing literature concerning fair LLMs. Specifically, a brief introduction to LLMs is provided, followed by an analysis of factors contributing to bias in LLMs. Additionally, the concept of fairness in LLMs is discussed categorically, summarizing metrics for evaluating bias in LLMs and existing algorithms for promoting fairness. Furthermore, resources for evaluating bias in LLMs, including toolkits and datasets, are summarized. Finally, existing research challenges and open questions are discussed.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). Builds on 145 of its 241 references that are in the map. First posted on arXiv 31 Mar 2024.

Builds on

145 of 241 references.

References · 145

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  11. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  12. 2017Deep reinforcement learning from human preferencescited by 3,846
38 more
  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  3. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  4. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  5. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  6. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  7. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  8. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  9. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  10. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  11. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  13. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  14. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  15. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  16. 2022Red Teaming Language Models with Language Modelscited by 983
  17. 2021Ethical and social risks of harm from Language Modelscited by 957
  18. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  19. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  20. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  21. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  22. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  23. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  24. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  25. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  26. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  27. 2021Fast Model Editing at Scalecited by 491
  28. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  29. 2018Gender Bias in Coreference Resolutioncited by 438
  30. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  31. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  32. 2023DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretrainingcited by 385
  33. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  34. 2023Editing Large Language Models: Problems, Methods, and Opportunitiescited by 367
  35. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  36. 2018Hypothesis Only Baselines in Natural Language Inferencecited by 291
  37. 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
  38. 2019Measuring Bias in Contextualized Word Representationscited by 275

50 most cited of 145 All 145 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 36; 2026: 8.

57 papers.

Sort:

By area:

  1. 2025Which of These Best Describes Multiple Choice Evaluation with LLMs? A) Forced B) Flawed C) Fixable D) All of the Abovecited by 51
  2. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  3. 2024The Impossibility of Fair LLMscited by 22
  4. 2025HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluationcited by 19
  5. 2024CEB: Compositional Evaluation Benchmark for Fairness in Large Language Modelscited by 17
  6. 2024FairAIED: Navigating Fairness, Bias, and Ethics in Educational AI Applicationscited by 15
  7. 2024Fairness Definitions in Language Models Explainedcited by 10
  8. 2024Fairness Testing of Large Language Models in Role-Playingcited by 10
  9. 2025Digital Forensics in the Age of Large Language Modelscited by 10
  10. 2024Beyond the Safety Bundle: Auditing the Helpful and Harmless Datasetcited by 8
  11. 2024Fairness in Large Language Models in Three Hourscited by 7
  12. 2025Building Trustworthy Multimodal AI: A Review of Fairness, Transparency, and Ethics in Vision-Language Taskscited by 7
38 more
  1. 2024Clinical Insights: A Comprehensive Review of Language Models in Medicinecited by 6
  2. 2024Improving LLM Group Fairness on Tabular Data via In-Context Learningcited by 6
  3. 2025Who's Asking? Investigating Bias Through the Lens of Disability Framed Queries in LLMscited by 6
  4. 2025Planted in Pretraining, Swayed by Finetuning: A Case Study on the Origins of Cognitive Biases in LLMscited by 5
  5. 2025CoBia: Constructed Conversations Can Trigger Otherwise Concealed Societal Biases in LLMscited by 5
  6. 2025Generative AI in Depth: A Survey of Recent Advances, Model Variants, and Real-World Applicationscited by 5
  7. 2025Probing LLM World Models: Enhancing Guesstimation with Wisdom of Crowds Decodingcited by 4
  8. 2025Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMscited by 4
  9. 2024AI-Driven Healthcare: A Review on Ensuring Fairness and Mitigating Biascited by 3
  10. 2025Fair Text Classification via Transferable Representationscited by 3
  11. 2025Enhancements for Developing a Comprehensive AI Fairness Assessment Standardcited by 3
  12. 2025On Fairness of Task Arithmetic: The Role of Task Vectorscited by 3
  13. 2025AMCR: A Framework for Assessing and Mitigating Copyright Risks in Generative Modelscited by 3
  14. 2025Towards Transparent AI: A Survey on Explainable Language Modelscited by 3
  15. 2024Rolling in the deep of cognitive and AI biasescited by 2
  16. 2025Reproducibility Study of "Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents"cited by 2
  17. 2025Is Your Model Fairly Certain? Uncertainty-Aware Fairness Evaluation for LLMscited by 2
  18. 2025Understanding Gender Bias in AI-Generated Product Descriptionscited by 2
  19. 2025The Disparate Impacts of Speculative Decodingcited by 2
  20. 2026When LLMs Imagine People: A Human-Centered Persona Brainstorm Audit for Bias and Fairness in Creative Applicationscited by 2
  21. 2024Exploring Accuracy-Fairness Trade-off in Large Language Modelscited by 1
  22. 2025The LLM Wears Prada: Analysing Gender Bias and Stereotypes through Online Shopping Datacited by 1
  23. 2025Iterative Multilingual Spectral Attribute Erasurecited by 1
  24. 2025Unified Threat Detection and Mitigation Framework (UTDMF): Combating Prompt Injection, Deception, and Bias in Enterprise-Scale Transformerscited by 1
  25. 2025Fairness-Aware Graph Representation Learning with Limited Demographic Informationcited by 1
  26. 2026Structurally Human, Semantically Biased: Detecting LLM-Generated References with Embeddings and GNNscited by 1
  27. 2025It's complicated. The relationship of algorithmic fairness and non-discrimination provisions for high-risk systems in the EU AI Actcited by 0
  28. 2025The Other Side of the Coin: Exploring Fairness in Retrieval-Augmented Generationcited by 0
  29. 2025Improving Fairness in LLMs Through Testing-Time Adversariescited by 0
  30. 2025Fairness Evaluation of Large Language Models in Academic Library Reference Servicescited by 0
  31. 2025Bridging Neural Networks and Dynamic Time Warping for Adaptive Time Series Classificationcited by 0
  32. 2025Exploring Causal Effect of Social Bias on Faithfulness Hallucinations in Large Language Modelscited by 0
  33. 2025FairFLRep: Fairness aware fault localization and repair of Deep Neural Networkscited by 0
  34. 2025A Close Reading Approach to Gender Narrative Biases in AI-Generated Storiescited by 0
  35. 2025BIPOLAR: Polarization-based granular framework for LLM bias evaluationcited by 0
  36. 2025Integrating gender inclusivity into large language models via instruction tuningcited by 0
  37. 2025Controllable Pareto Trade-off between Fairness and Accuracycited by 0
  38. 2025Bias Testing and Mitigation in Black Box LLMs using Metamorphic Relationscited by 0

50 most cited of 57 All 57 (JSON)