# Cultural Bias in Large Language Models: Evaluating AI Agents through Moral Questionnaires

arXiv: https://arxiv.org/abs/2507.10073 · HTML: https://mlsanity.com/paper/2507.10073/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Social simulation and multi-agent debate, one of 16 directions in LLM agents and tool use.
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 14 Jul 2025.

## Placed in

- LLM agents and tool use / [Social simulation and multi-agent debate](https://mlsanity.com/agents/social-simulation-and-multi-agent-debate/)

## Builds on (most cited first, 12 of 12)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Out of One, Many: Using Language Models to Simulate Human Samples](https://mlsanity.com/paper/2209.06899/) (2022)
- [Faith and Fate: Limits of Transformers on Compositionality](https://mlsanity.com/paper/2305.18654/) (2023)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://mlsanity.com/paper/2005.00813/) (2020)
- [Simulacra as Conscious Exotica](https://mlsanity.com/paper/2402.12422/) (2024)
- [Political Bias in LLMs: Unaligned Moral Values in Agent-centric Simulations](https://mlsanity.com/paper/2408.11415/) (2024)

## Built on by (most cited first, 4 of 4)

- [The Fragility Of Moral Judgment In Large Language Models](https://mlsanity.com/paper/2603.05651/) (2026)
- [EvalMORAAL: Interpretable Chain-of-Thought and LLM-as-Judge Evaluation for Moral Alignment in Large Language Models](https://mlsanity.com/paper/2510.05942/) (2025)
- [Mitigating Cultural Bias in LLMs via Multi-Agent Cultural Debate](https://mlsanity.com/paper/2601.12091/) (2026)
- [Overton Pluralistic Reinforcement Learning for Large Language Models](https://mlsanity.com/paper/2602.20759/) (2026)

## Abstract (arXiv)

Are AI systems truly representing human values, or merely averaging across them? Our study suggests a concerning reality: Large Language Models (LLMs) fail to represent diverse cultural moral frameworks despite their linguistic capabilities. We expose significant gaps between AI-generated and human moral intuitions by applying the Moral Foundations Questionnaire across 19 cultural contexts. Comparing multiple state-of-the-art LLMs' origins against human baseline data, we find these models systematically homogenize moral diversity. Surprisingly, increased model size doesn't consistently improve cultural representation fidelity. Our findings challenge the growing use of LLMs as synthetic populations in social science research and highlight a fundamental limitation in current AI alignment approaches. Without data-driven alignment beyond prompting, these systems cannot capture the nuanced, culturally-specific moral intuitions. Our results call for more grounded alignment objectives and evaluation metrics to ensure AI systems represent diverse human values rather than flattening the moral landscape.