mlsanityData to 7 Oct 2026

Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models

arXiv 2304.03738 ↗First posted 7 Apr 2023cs.CY cs.CLcited by 144

Not on the map. Why?

Abstract (arXiv)

As the capabilities of generative language models continue to advance, the implications of biases ingrained within these models have garnered increasing attention from researchers, practitioners, and the broader public. This article investigates the challenges and risks associated with biases in large-scale language models like ChatGPT. We discuss the origins of biases, stemming from, among others, the nature of training data, model specifications, algorithmic constraints, product design, and policy decisions. We explore the ethical concerns arising from the unintended consequences of biased model outputs. We further analyze the potential opportunities to mitigate biases, the inevitability of some biases, and the implications of deploying these models in various applications, such as virtual assistants, content generation, and chatbots. Finally, we review the current approaches to identify, quantify, and mitigate biases in language models, emphasizing the need for a multi-disciplinary, collaborative effort to develop more equitable, transparent, and responsible AI systems. This article aims to stimulate a thoughtful dialogue within the artificial intelligence community, encouraging researchers and developers to reflect on the role of biases in generative language models and the ongoing pursuit of ethical AI.

Key facts. Cited by 144 papers in the map (data through 7 Oct 2026). Builds on 74 of its 163 references that are in the map. First posted on arXiv 7 Apr 2023.

Builds on

74 of 163 references.

References · 74

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  11. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  12. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
38 more
  1. 2022Emergent Abilities of Large Language Modelscited by 2,483
  2. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  3. 2020Extracting Training Data from Large Language Modelscited by 2,144
  4. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  5. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  6. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  7. 2020Don't Stop Pretraining: Adapt Language Models to Domains and Taskscited by 1,548
  8. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
  9. 2018Datasheets for Datasetscited by 1,371
  10. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  11. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  12. 2018Model Cards for Model Reportingcited by 1,068
  13. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 979
  14. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  15. 2021Ethical and social risks of harm from Language Modelscited by 957
  16. 2019How Can We Know What Language Models Know?cited by 953
  17. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  18. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  19. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  20. 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 864
  21. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
  22. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  23. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  24. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  25. 2019How multilingual is Multilingual BERT?cited by 642
  26. 2019Green AIcited by 620
  27. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 582
  28. 2019Defending Against Neural Fake Newscited by 553
  29. 2019Release Strategies and the Social Impacts of Language Modelscited by 505
  30. 2017Word Translation Without Parallel Datacited by 480
  31. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  32. 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
  33. 2016Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatmentcited by 368
  34. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 343
  35. 2018Actionable Recourse in Linear Classificationcited by 327
  36. 2019Learning Deep Transformer Models for Machine Translationcited by 309
  37. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  38. 2017Recent Trends in Deep Learning Based Natural Language Processingcited by 262

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 43; 2024: 58; 2025: 33; 2026: 10.

144 papers.

Sort:

By area:

  1. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  2. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  3. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  4. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  5. 2023A Survey on Large Language Models for Recommendationcited by 378
  6. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 354
  7. 2023Challenges and Applications of Large Language Modelscited by 251
  8. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Usecited by 209
  9. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 176
  10. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  11. 2023Cultural Bias and Cultural Alignment of Large Language Modelscited by 149
  12. 2023Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injectioncited by 143
38 more
  1. 2023Factuality Challenges in the Era of Large Language Modelscited by 125
  2. 2023DataInf: Efficiently Estimating Data Influence in LoRA-tuned LLMs and Diffusion Modelscited by 121
  3. 2023Generative AIcited by 119
  4. 2024Emergent social conventions and collective bias in LLM populationscited by 115
  5. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  6. 2023Learning Linear Causal Representations from Interventions under General Nonlinear Mixingcited by 92
  7. 2023A Survey on Fairness in Large Language Modelscited by 91
  8. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  9. 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Biascited by 82
  10. 2024Bias in Large Language Models: Origin, Evaluation, and Mitigationcited by 77
  11. 2025LLM Generated Persona is a Promise with a Catchcited by 77
  12. 2024Risk Taxonomy, Mitigation, and Assessment Benchmarks of Large Language Model Systemscited by 63
  13. 2024Understanding LLMs: A Comprehensive Overview from Training to Inferencecited by 58
  14. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  15. 2024Language Repository for Long Video Understandingcited by 55
  16. 2024A Comprehensive Survey of Large Language Models and Multimodal Large Language Models in Medicinecited by 45
  17. 2024IrokoBench: A New Benchmark for African Languages in the Age of Large Language Modelscited by 43
  18. 2023The Unequal Opportunities of Large Language Models: Revealing Demographic Bias through Job Recommendationscited by 41
  19. 2023GenAI Against Humanity: Nefarious Applications of Generative Artificial Intelligence and Large Language Modelscited by 40
  20. 2023AD-AutoGPT: An Autonomous GPT for Alzheimer's Disease Infodemiologycited by 34
  21. 2024A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directionscited by 34
  22. 2023Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectivescited by 31
  23. 2023A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Timecited by 30
  24. 2025Do Large Language Models Solve the Problems of Agent-Based Modeling? A Critical Review of Generative Social Simulationscited by 28
  25. 2024Data-Centric Foundation Models in Computational Healthcare: A Surveycited by 27
  26. 2025Exploring the Roles of Large Language Models in Reshaping Transportation Systems: A Survey, Framework, and Roadmapcited by 26
  27. 2023The Moral Machine Experiment on Large Language Modelscited by 24
  28. 2025The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMscited by 24
  29. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  30. 2025Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judgecited by 23
  31. 2023Bias and Fairness in Chatbots: An Overviewcited by 20
  32. 2024DeLLMa: Decision Making Under Uncertainty with Large Language Modelscited by 20
  33. 2023Automated Claim Matching with Large Language Models: Empowering Fact-Checkers in the Fight Against Misinformationcited by 18
  34. 2024Few-Shot Fairness: Unveiling LLM's Potential for Fairness-Aware Classificationcited by 18
  35. 2024A Comprehensive Survey on Evaluating Large Language Model Applications in the Medical Industrycited by 17
  36. 2024CEB: Compositional Evaluation Benchmark for Fairness in Large Language Modelscited by 17
  37. 2023ChatGPT is a Remarkable Tool -- For Expertscited by 16
  38. 2023TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varietiescited by 16

50 most cited of 144 All 144 (JSON)