mlsanityData to 7 Oct 2026

Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPT

arXiv 2405.06996 ↗First posted 11 May 2024cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

While nationality is a pivotal demographic element that enhances the performance of language models, it has received far less scrutiny regarding inherent biases. This study investigates nationality bias in ChatGPT (GPT-3.5), a large language model (LLM) designed for text generation. The research covers 195 countries, 4 temperature settings, and 3 distinct prompt types, generating 4,680 discourses about nationality descriptions in Chinese and English. Automated metrics were used to analyze the nationality bias, and expert annotators alongside ChatGPT itself evaluated the perceived bias. The results show that ChatGPT's generated discourses are predominantly positive, especially compared to its predecessor, GPT-2. However, when prompted with negative inclinations, it occasionally produces negative content. Despite ChatGPT considering its generated text as neutral, it shows consistent self-awareness about nationality bias when subjected to the same pair-wise comparison annotation framework used by human annotators. In conclusion, while ChatGPT's generated texts seem friendly and positive, they reflect the inherent nationality biases in the real world. This bias may vary across different language versions of ChatGPT, indicating diverse cultural perspectives. The study highlights the subtle and pervasive nature of biases within LLMs, emphasizing the need for further scrutiny.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 35 of its 49 references that are in the map. First posted on arXiv 11 May 2024.

Builds on

35 of 49 references.

References · 35

  1. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  2. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  3. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,235
  4. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  5. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  6. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  7. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  8. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  9. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  10. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  11. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  12. 2018Gender Bias in Coreference Resolutioncited by 438
23 more
  1. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  2. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 376
  3. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  4. 2019Measuring Bias in Contextualized Word Representationscited by 275
  5. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  6. 2020Learning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detectioncited by 217
  7. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  8. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 191
  9. 2022"I'm sorry to hear that": Finding New Biases in Language Models with a Holistic Descriptor Datasetcited by 160
  10. 2018Reducing Gender Bias in Abusive Language Detectioncited by 152
  11. 2021Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Modelscited by 141
  12. 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
  13. 2019Assessing Social and Intersectional Biases in Contextualized Word Representationscited by 134
  14. 2018Assessing Gender Bias in Machine Translation -- A Case Study with Google Translatecited by 121
  15. 2019Toward Gender-Inclusive Coreference Resolutioncited by 84
  16. 2022COLD: A Benchmark for Chinese Offensive Language Detectioncited by 74
  17. 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
  18. 2019On Measuring Gender Bias in Translation of Gender-neutral Pronounscited by 59
  19. 2020Racism is a Virus: Anti-Asian Hate and Counterspeech in Social Media during the COVID-19 Crisiscited by 40
  20. 2022Towards Identifying Social Bias in Dialog Systems: Frame, Datasets, and Benchmarkscited by 36
  21. 2020Quantifying Intimacy in Languagecited by 25
  22. 2022On Xing Tian and the Perseverance of Anti-China Sentiment Onlinecited by 10
  23. 2022Towards Robust NLG Bias Evaluation with Syntactically-diverse Promptscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 10; 2026: 1.

13 papers.

Sort:

By area:

  1. 2024Exploring Changes in Nation Perception with Nationality-Assigned Personas in LLMscited by 11
  2. 2025Colombian Waitresses y Jueces canadienses: Gender and Country Biases in Occupation Recommendations from LLMscited by 7
  3. 2025Echoes of Power: Investigating Geopolitical Bias in US and China Large Language Modelscited by 5
  4. 2025No LLM is Free From Bias: A Comprehensive Study of Bias Evaluation in Large Language Modelscited by 3
  5. 2025LocalBench: Benchmarking LLMs on County-Level Local Knowledge and Reasoningcited by 3
  6. 2024Desert Camels and Oil Sheikhs: Arab-Centric Red Teaming of Frontier LLMscited by 1
  7. 2025Spot the BlindSpots: Systematic Identification and Quantification of Fine-Grained LLM Biases in Contact Center Summariescited by 1
  8. 2025Social Bias in Multilingual Language Models: A Surveycited by 1
  9. 2025Assessing Agentic Large Language Models in Multilingual National Biascited by 0
  10. 2025Measuring Stereotype and Deviation Biases in Large Language Modelscited by 0
  11. 2025Measuring Algorithmic Partisanship via Zero-Shot Classification and Its Implications on Political Discoursecited by 0
  12. 2025Collective Narrative Grounding: Community-Coordinated Data Contributions to Improve Local AI Systemscited by 0
1 more
  1. 2026When Who You Are Can Change the Code You Get: A Study of Persona-Induced Bias in LLM Code Generationcited by 0