# Toward Cultural Alignment: Human-Centered Evaluation of Multimodal AI Stories Across Five African Communities

arXiv: https://arxiv.org/abs/2608.29209 · HTML: https://mlsanity.com/paper/2608.29209/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 29 Aug 2026.

## Builds on (most cited first, 22 of 22)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [Evaluating the Moral Beliefs Encoded in LLMs](https://mlsanity.com/paper/2307.14324/) (2023)
- [Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Study](https://mlsanity.com/paper/2303.17466/) (2023)
- [Towards Measuring and Modeling "Culture" in LLMs: A Survey](https://mlsanity.com/paper/2403.15412/) (2024)
- [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022)
- [Are Large Language Models Capable of Generating Human-Level Narratives?](https://mlsanity.com/paper/2407.13248/) (2024)
- [AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances](https://mlsanity.com/paper/2409.11360/) (2024)
- [Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs](https://mlsanity.com/paper/2503.08688/) (2025)
- [NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models](https://mlsanity.com/paper/2404.12464/) (2024)
- [NormDial: A Comparable Bilingual Synthetic Dialog Dataset for Modeling Social Norm Adherence and Violation](https://mlsanity.com/paper/2310.14563/) (2023)
- [CulturalFrames: Assessing Cultural Expectation Alignment in Text-to-Image Models and Evaluation Metrics](https://mlsanity.com/paper/2506.08835/) (2025)
- [RENOVI: A Benchmark Towards Remediating Norm Violations in Socio-Cultural Conversations](https://mlsanity.com/paper/2402.11178/) (2024)
- [Does Cross-Cultural Alignment Change the Commonsense Morality of Language Models?](https://mlsanity.com/paper/2406.16316/) (2024)
- [Fluent but Foreign: Even Regional LLMs Lack Cultural Alignment](https://mlsanity.com/paper/2505.21548/) (2025)
- [Culture in Action: Evaluating Text-to-Image Models through Social Activities](https://mlsanity.com/paper/2511.05681/) (2025)
- [TALES: A Taxonomy and Analysis of Cultural Representations in LLM-generated Stories](https://mlsanity.com/paper/2511.21322/) (2025)
- [CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systems](https://mlsanity.com/paper/2506.08071/) (2025)
- [Cultural Fidelity in Large-Language Models: An Evaluation of Online Language Resources as a Driver of Model Performance in Value Representation](https://mlsanity.com/paper/2410.10489/) (2024)
- [Evaluating AI-Generated Images of Cultural Artifacts with Community-Informed Rubrics](https://mlsanity.com/paper/2604.02406/) (2026)

## Abstract (arXiv)

In this paper, we examine how well AI-generated multimodal stories align with the lived practices, relationships, language, values, and visual expectations of the communities they represent. We conduct a community-grounded mixed-methods evaluation with 19 culture representatives across five African communities, combining quantitative annotations with qualitative focus group discussions. We find that cultural alignment depends not simply on recognizable cultural markers, but on how those markers fit social, linguistic, procedural, and visual context. From these evaluations, we develop a taxonomy of cultural alignment comprising five broader cultural marker categories and eight recurring mechanisms of misalignment. We additionally evaluate five multimodal LLM judges to examine whether automated evaluation can approximate community-grounded judgments at scale. Judge reliability and score calibration vary substantially across communities, with no single judge performing consistently across all five settings. These findings motivate community-calibrated evaluation pipelines in which automated judges are validated against community judgments to determine where they can be trusted and where human review remains necessary.