Anchoring LLM Gender Bias to Human Baselines: A Cross-Lingual Audit
Not on the map. Why?
Abstract (arXiv)
We audit six large language models (LLMs) for gender stereotyping across English, Korean, Chinese, and Japanese. Three were developed primarily for English-language use (Claude, GPT, Gemini) and three for East Asian use (DeepSeek, Syn-Pro, HyperCLOVA X). We adopt the HEXACO-100 personality inventory and anchor each model against a cross-cultural human dataset spanning 48 countries to ask not whether LLMs are biased, but how far their gender attributions drift from the populations they are deployed among. Our findings show that their stereotyping spans a range roughly 2.5 times wider than the entire cross-country range found in humans, and the effect can compound across languages. One English-centric model, prompted in Korean, reached 5 times the local baseline, even when the prompt stated the candidate had already been hired, which often dampens human stereotyping. To characterize such behaviors without ranking them, we introduce a four-pattern framework -- concordance, suppression, reorganization, and amplification -- across 24 (model x language) cells. Item-level analysis reveals that translation does not just rescale stereotypes, but changes the attributes tied to it, hiding significant rearrangement under the surface while appearing well-calibrated. Our results ultimately suggest that no single debiasing pipeline is likely to address bias evenly across linguistic boundaries.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 52 references that are in the map. First posted on arXiv 29 May 2026.
Builds on
12 of 52 references.
References · 12
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
- 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
- 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
- 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 701
- 2018Gender Bias in Coreference Resolutioncited by 442
- 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 412
- 2024Gender, Race, and Intersectional Bias in Resume Screening via Language Model Retrievalcited by 51
- 2023Large Language Models as Superpositions of Cultural Perspectivescited by 49
- 2024JobFair: A Framework for Benchmarking Gender Hiring Bias in Large Language Modelscited by 21
- 2023Large language models can replicate cross-cultural differences in personalitycited by 8
- 2026Personality Shapes Gender Bias in Persona-Conditioned LLM Narratives Across English and Hindi: An Empirical Investigationcited by 1
Built on by
Not cited yet.