# CCBENCH: Assessing LLM Cultural Competence via Implicitly Signaled Norms using Health Queries

arXiv: https://arxiv.org/abs/2607.05405 · HTML: https://mlsanity.com/paper/2607.05405/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 49 references that are in the map.
- First posted on arXiv 8 Jun 2026.

## Builds on (most cited first, 30 of 30)

- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [Towards Measuring the Representation of Subjective Global Opinions in Language Models](https://mlsanity.com/paper/2306.16388/) (2023)
- [Global MMLU: Understanding and Addressing Cultural and Linguistic Biases in Multilingual Evaluation](https://mlsanity.com/paper/2412.03304/) (2024)
- [BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages](https://mlsanity.com/paper/2406.09948/) (2024)
- [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022)
- [Cultural Bias and Cultural Alignment of Large Language Models](https://mlsanity.com/paper/2311.14096/) (2023)
- [Investigating Cultural Alignment of Large Language Models](https://mlsanity.com/paper/2402.13231/) (2024)
- [Generative Language Models Exhibit Social Identity Biases](https://mlsanity.com/paper/2310.15819/) (2023)
- [Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries](https://mlsanity.com/paper/2310.13132/) (2023)
- [Unintended Impacts of LLM Alignment on Global Representation](https://mlsanity.com/paper/2402.15018/) (2024)
- [NormBank: A Knowledge Bank of Situational Social Norms](https://mlsanity.com/paper/2305.17008/) (2023)
- [Randomness, Not Representation: The Unreliability of Evaluating Cultural Alignment in LLMs](https://mlsanity.com/paper/2503.08688/) (2025)
- [Knowledge of cultural moral norms in large language models](https://mlsanity.com/paper/2306.01857/) (2023)
- [NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models](https://mlsanity.com/paper/2404.12464/) (2024)
- [Culture is Not Trivia: Sociocultural Theory for Cultural NLP](https://mlsanity.com/paper/2502.12057/) (2025)
- [Multilingual != Multicultural: Evaluating Gaps Between Multilingual Capabilities and Cultural Alignment in LLMs](https://mlsanity.com/paper/2502.16534/) (2025)
- [Can Language Models Reason about Individualistic Human Values and Preferences?](https://mlsanity.com/paper/2410.03868/) (2024)
- [Stereotype or Personalization? User Identity Biases Chatbot Recommendations](https://mlsanity.com/paper/2410.05613/) (2024)
- [Beyond Aesthetics: Cultural Competence in Text-to-Image Models](https://mlsanity.com/paper/2407.06863/) (2024)
- [The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Support](https://mlsanity.com/paper/2401.14362/) (2024)
- [Extrinsic Evaluation of Cultural Competence in Large Language Models](https://mlsanity.com/paper/2406.11565/) (2024)
- [Normative Evaluation of Large Language Models with Everyday Moral Dilemmas](https://mlsanity.com/paper/2501.18081/) (2025)
- [Reading Between the Prompts: How Stereotypes Shape LLM's Implicit Personalization](https://mlsanity.com/paper/2505.16467/) (2025)
- [CLAVE: An Adaptive Framework for Evaluating Values of LLM Generated Responses](https://mlsanity.com/paper/2407.10725/) (2024)
- [From Personas to Talks: Revisiting the Impact of Personas on LLM-Synthesized Emotional Support Conversations](https://mlsanity.com/paper/2502.11451/) (2025)
- [Presumed Cultural Identity: How Names Shape LLM Responses](https://mlsanity.com/paper/2502.11995/) (2025)
- [Rejected Dialects: Biases Against African American Language in Reward Models](https://mlsanity.com/paper/2502.12858/) (2025)
- [Cultural Learning-Based Culture Adaptation of Language Models](https://mlsanity.com/paper/2504.02953/) (2025)
- [Tailored Emotional LLM-Supporter: Enhancing Cultural Sensitivity](https://mlsanity.com/paper/2508.07902/) (2025)
- [Cultural Compass: A Framework for Organizing Societal Norms to Detect Violations in Human-AI Conversations](https://mlsanity.com/paper/2601.07973/) (2026)

## Abstract (arXiv)

To interact with users fairly and without stereotyping, AI models must display cultural competency, i.e., the ability to infer and adapt to a user's implicitly signaled cultural values, rather than relying on static demographic traits. We introduce CCBENCH, a framework for evaluating cultural competency in large language models (LLMs), treating culture as a continuum of norm adherence states rather than as a binary state of cultural belongingness. As a case study on health, we create CCBENCH-Health, which includes 60 theoretically grounded personas exhibiting varied norm-adherence states across six cultures, each engaging in 18 realistic dialogues. Each persona is evaluated on 52 authentic healthcare questions drawn from real user forums, yielding 3,120 unique interactions. Benchmarking five leading models reveals that even the best achieve culturally appropriate responses only 20-30% of the time. When explicitly prompted to focus on culturally relevant cues from the conversational history (CoT), performance improves modestly by 3-5% on average. We find that models perform best when personas avoid cultural norms rather than follow them, revealing a persistent asymmetry, suggesting a preference in the models to align with built-in biases than adapt to cultural cues. This is especially observed in the Afghan context (Avg: 8.8%), where cultural cues rarely yield appropriate health advice. Finally, we find that models sometimes adapt more readily to implicit, cultural conversational styles than to explicitly stated cultural practices, though this varies across cultures.