mlsanityData to 7 Oct 2026

A Survey on Fairness in Large Language Models

arXiv 2308.10149 ↗First posted 20 Aug 2023cs.CL cs.AIcited by 91

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have shown powerful performance and development prospects and are widely deployed in the real world. However, LLMs can capture social biases from unprocessed training data and propagate the biases to downstream tasks. Unfair LLM systems have undesirable social impacts and potential harms. In this paper, we provide a comprehensive review of related research on fairness in LLMs. Considering the influence of parameter magnitude and training paradigm on research strategy, we divide existing fairness research into oriented to medium-sized LLMs under pre-training and fine-tuning paradigms and oriented to large-sized LLMs under prompting paradigms. First, for medium-sized LLMs, we introduce evaluation metrics and debiasing methods from the perspectives of intrinsic bias and extrinsic bias, respectively. Then, for large-sized LLMs, we introduce recent fairness research, including fairness evaluation, reasons for bias, and debiasing methods. Finally, we discuss and provide insight on the challenges and future directions for the development of fairness in LLMs.

Key facts. Cited by 91 papers in the map (data through 7 Oct 2026). Builds on 133 of its 179 references that are in the map. First posted on arXiv 20 Aug 2023.

Builds on

133 of 179 references.

References · 133

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  10. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  11. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  12. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
38 more
  1. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  5. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  7. 2022Emergent Abilities of Large Language Modelscited by 2,483
  8. 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
  9. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  10. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  11. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  12. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  13. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  14. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  15. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  16. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  17. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  18. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  19. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  20. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  21. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 557
  22. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  23. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 505
  24. 2020UnifiedQA: Crossing Format Boundaries With a Single QA Systemcited by 475
  25. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  26. 2018Gender Bias in Coreference Resolutioncited by 438
  27. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  28. 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 403
  29. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  30. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  31. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  32. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  33. 2019Measuring Bias in Contextualized Word Representationscited by 275
  34. 2020Social Chemistry 101: Learning to Reason about Social and Moral Normscited by 253
  35. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 251
  36. 2020Fairness without Demographics through Adversarially Reweighted Learningcited by 251
  37. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 236
  38. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234

50 most cited of 133 All 133 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 38; 2025: 31; 2026: 17.

91 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  2. 2023Explainability for Large Language Models: A Surveycited by 331
  3. 2023Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMscited by 191
  4. 2023A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethicscited by 162
  5. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  6. 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 157
  7. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 117
  8. 2024Bias in Large Language Models: Origin, Evaluation, and Mitigationcited by 77
  9. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  10. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Textscited by 49
  11. 2024FairMedFM: Fairness Benchmarking for Medical Imaging Foundation Modelscited by 47
  12. 2024Thinking Fair and Slow: On the Efficacy of Structured Prompts for Debiasing Language Modelscited by 44
38 more
  1. 2024When Large Language Models Meet Vector Databases: A Surveycited by 39
  2. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  3. 2024Navigating LLM Ethics: Advancements, Challenges, and Future Directionscited by 28
  4. 2024Bias patterns in the application of LLMs for clinical decision support: A comprehensive studycited by 27
  5. 2024Investigating Implicit Bias in Large Language Models: A Large-Scale Study of Over 50 LLMscited by 25
  6. 2025SafeDialBench: A Fine-Grained Safety Evaluation Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attackscited by 23
  7. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  8. 2024Towards Trustworthy AI: A Review of Ethical and Robust Large Language Modelscited by 21
  9. 2024Surveying the MLLM Landscape: A Meta-Review of Current Surveyscited by 20
  10. 2026Large Language Model Reasoning Failurescited by 20
  11. 2024Causal Evaluation of Language Modelscited by 18
  12. 2024Large Action Models: From Inception to Implementationcited by 18
  13. 2024CEB: Compositional Evaluation Benchmark for Fairness in Large Language Modelscited by 17
  14. 2025When Your AI Agent Succumbs to Peer-Pressure: Studying Opinion-Change Dynamics of LLMscited by 14
  15. 2024Are Models Biased on Text without Gender-related Language?cited by 10
  16. 2024A survey on fairness of large language models in e-commerce: progress, application, and challengecited by 10
  17. 2024Representation Bias in Political Sample Simulations with Large Language Modelscited by 10
  18. 2024Your Large Language Model is Secretly a Fairness Proponent and You Should Prompt it Like Onecited by 9
  19. 2024Certifying Counterfactual Bias in LLMscited by 9
  20. 2025Orchestrating Agents and Data for Enterprise: A Blueprint Architecture for Compound AIcited by 9
  21. 2023Understanding Fairness Surrogate Functions in Algorithmic Fairnesscited by 8
  22. 2025Disparities in LLM Reasoning Accuracy and Explanations: A Case Study on African American Englishcited by 8
  23. 2024Unveiling Bias in Fairness Evaluations of Large Language Models: A Critical Literature Review of Music and Movie Recommendation Systemscited by 7
  24. 2024Word Importance Explains How Prompts Affect Language Model Outputscited by 6
  25. 2025Rethinking LLM Bias Probing Using Lessons from the Social Sciencescited by 6
  26. 2025Concept-Level Explainability for Auditing & Steering LLM Responsescited by 6
  27. 2025Understanding the Anchoring Effect of LLM with Synthetic Data: Existence, Mechanism, and Potential Mitigationscited by 6
  28. 2026CatRAG: Functor-Guided Structural Debiasing with Retrieval Augmentation for Fair LLMscited by 6
  29. 2024Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forwardcited by 5
  30. 2024Order-Independence Without Fine Tuningcited by 5
  31. 2024Mechanistic interpretability of large language models with applications to the financial services industrycited by 5
  32. 2025Evaluate Bias without Manual Test Sets: A Concept Representation Perspective for LLMscited by 5
  33. 2024Challenges in Mechanistically Interpreting Model Representationscited by 4
  34. 2024BTBR: A Bayesian-Theory-Driven Probabilistic-Fuzzy Framework for Implicit Bias Removal in Large Language Modelscited by 4
  35. 2025Who Owns the Output? Bridging Law and Technology in LLMs Attributioncited by 4
  36. 2025Modular Machine Learning: An Indispensable Path towards New-Generation Large Language Modelscited by 4
  37. 2025Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMscited by 4
  38. 2024Survey of Natural Language Processing for Education: Taxonomy, Systematic Review, and Future Trendscited by 3

50 most cited of 91 All 91 (JSON)