# Probing Cultural Awareness in LLMs: A Case Study of Cross-Culture Aesthetic Stylistics

arXiv: https://arxiv.org/abs/2605.27296 · HTML: https://mlsanity.com/paper/2605.27296/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 26 May 2026.

## Builds on (most cited first, 13 of 13)

- [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://mlsanity.com/paper/2010.00133/) (2020)
- [Survey of Cultural Awareness in Language Models: Text and Beyond](https://mlsanity.com/paper/2411.00860/) (2024)
- [LLM as a Mastermind: A Survey of Strategic Reasoning with Large Language Models](https://mlsanity.com/paper/2404.01230/) (2024)
- [CultureLLM: Incorporating Cultural Differences into Large Language Models](https://mlsanity.com/paper/2402.10946/) (2024)
- [Does Mapo Tofu Contain Coffee? Probing LLMs for Food-related Cultural Knowledge](https://mlsanity.com/paper/2404.06833/) (2024)
- [Instruct Once, Chat Consistently in Multiple Rounds: An Efficient Tuning Framework for Dialogue](https://mlsanity.com/paper/2402.06967/) (2024)
- [KddRES: A Multi-level Knowledge-driven Dialogue Dataset for Restaurant Towards Customized Dialogue System](https://mlsanity.com/paper/2011.08772/) (2020)
- [Cultural Adaptation of Recipes](https://mlsanity.com/paper/2310.17353/) (2023)
- [Learning to Correct for QA Reasoning with Black-box LLMs](https://mlsanity.com/paper/2406.18695/) (2024)
- [Hanfu-Bench: A Multimodal Benchmark on Cross-Temporal Cultural Understanding and Transcreation](https://mlsanity.com/paper/2506.01565/) (2025)
- [Do Large Language Models Understand Morality Across Cultures?](https://mlsanity.com/paper/2507.21319/) (2025)
- [Tracing Semantic Variation in Slang](https://mlsanity.com/paper/2210.08635/) (2022)
- [How Well Do LLMs Handle Cantonese? Benchmarking Cantonese Capabilities of Large Language Models](https://mlsanity.com/paper/2408.16756/) (2024)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly deployed in diverse cultural contexts, yet their ability to master aesthetic stylistics, i.e., the strategic use of language to evoke cultural resonance, remains underexplored. We curate C4STYLI, a benchmark of highly stylized translated movie titles and advertising slogans from Hong Kong and the Chinese Mainland, to evaluate LLMs via the lens of behavioral recognition and productive competence. Extensive evaluations show that LLMs differ from humans in stylistic recognition, and this recognition ability varies across text domains. In addition, stylistic recognition and generation performance in LLMs are not consistently aligned. To further examine whether LLMs genuinely capture stylistic information in stylistic recognition, we conduct structural ablation with logistic regression probes. We find that, in the Hong Kong setting, stylistic recognition in LLMs relies primarily on surface-level linguistic information rather than stylistic structure. This suggests limited sensitivity to Hong Kong-specific stylistic structure.