mlsanityData to 14 Oct 2026

Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnaires

arXiv 2507.10073 ↗First posted 14 Jul 2025cs.CL cs.AIcited by 4

On the map: LLM agents and tool use → Social simulation and multi-agent debate

Abstract (arXiv)

Are AI systems truly representing human values, or merely averaging across them? Our study suggests a concerning reality: Large Language Models (LLMs) fail to represent diverse cultural moral frameworks despite their linguistic capabilities. We expose significant gaps between AI-generated and human moral intuitions by applying the Moral Foundations Questionnaire across 19 cultural contexts. Comparing multiple state-of-the-art LLMs' origins against human baseline data, we find these models systematically homogenize moral diversity. Surprisingly, increased model size doesn't consistently improve cultural representation fidelity. Our findings challenge the growing use of LLMs as synthetic populations in social science research and highlight a fundamental limitation in current AI alignment approaches. Without data-driven alignment beyond prompting, these systems cannot capture the nuanced, culturally-specific moral intuitions. Our results call for more grounded alignment objectives and evaluation metrics to ensure AI systems represent diverse human values rather than flattening the moral landscape.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use. Builds on 12 of its 24 references that are in the map. First posted on arXiv 14 Jul 2025.

Builds on

12 of 24 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Political Bias in LLMs: Unaligned Moral Values in Agent-centric Simulations21 Aug
  2. 2023Generative Agents: Interactive Simulacra of Human Behavior7 Apr

Other references · 10

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2024Qwen2.5 Technical Reportcited by 4,942
  3. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  4. 2023Mistral 7Bcited by 3,164
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  6. 2022Out of One, Many: Using Language Models to Simulate Human Samplescited by 752
  7. 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 503
  8. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  9. 2020Social Biases in NLP Models as Barriers for Persons with Disabilitiescited by 182
  10. 2024Simulacra as Conscious Exoticacited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026The Fragility Of Moral Judgment In Large Language Modelscited by 8
  2. 2025EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Modelscited by 6
  3. 2026Mitigating Cultural Bias in LLMs via Multi-Agent Cultural Debatecited by 1
  4. 2026Overton Pluralistic Reinforcement Learning for Large Language Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025One Model, Many Morals: Uncovering Cross-Linguistic Misalignments in Computational Moral Reasoning
  2. 2026Toward a Theory of Value in AI Alignment
  3. 2024ValueCompass: A Framework for Measuring Contextual Value Alignment Between Human and LLMs
  4. 2024Whose Morality Do They Speak? Unraveling Cultural Bias in Multilingual Language Models
  5. 2025The Morality of Probability: How Implicit Moral Biases in LLMs May Shape the Future of Human-AI Symbiosis
  6. 2024Language Model Alignment in Multilingual Trolley Problems
  7. 2025Diverse Human Value Alignment for Large Language Models via Ethical Reasoning
  8. 2023Cultural Bias and Cultural Alignment of Large Language Models