mlsanityData to 14 Oct 2026

Do LLMs Use Cultural Knowledge Without Being Told? A Multilingual Evaluation of Implicit Pragmatic Adaptation

arXiv 2604.17718 ↗First posted 20 Apr 2026cs.CL cs.SIcited by 0

Not on the map. Why?

Abstract (arXiv)

Many benchmarks show that large language models can answer direct questions about culture. We study a different question: do they also change how they speak when culture is only implied by the situation? We evaluate 60 culturally grounded conversational scenarios across five languages in three conditions: a neutral baseline (Prompt A), an explicit cultural instruction (Prompt B), and implicit situational cueing (Prompt C). We score responses on 12 pragmatic features covering deference to authority, individual-versus-group framing, and uncertainty management. We define Pragmatic Context Sensitivity (PCS) as the fraction of the Prompt A->B shift that reappears under Prompt A->C. Across four deployed LLMs and five languages (English, German, Hindi, Nepali, Urdu), the primary stable-only PCS mean is 0.196 (SD = 0.113), indicating that the models recover only about one-fifth of the pragmatic shift they can produce when instructed explicitly. Transfer is strongest for authority-related cues (0.299) and weakest for individual-versus-group framing (0.120). Uncertainty-related behaviour is mixed: hedging density exhibits negative explicit gaps in all five languages, suggesting that alignment training actively suppresses the target behaviour. Because Hindi and Urdu share core grammar yet index distinct cultural communities, we use them as a natural control; a paired analysis finds no reliable baseline difference (t = 0.96, p = 0.339, dz = 0.06), suggesting that models respond primarily to linguistic structure rather than to the cultural associations a language carries. We argue that multilingual cultural pragmatics is an explicit-versus-implicit deployment problem, not only a factual knowledge problem.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 15 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

8 of 15 references.

References · 8

  1. 2020XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalizationcited by 742
  2. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  3. 2023MEGA: Multilingual Evaluation of Generative AIcited by 293
  4. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 206
  5. 2022A fine-grained comparison of pragmatic language understanding in humans and language modelscited by 70
  6. 2022The Goldilocks of Pragmatic Understanding: Fine-Tuning Strategy Matters for Implicature Resolution by LLMscited by 58
  7. 2024NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Modelscited by 48
  8. 2023EtiCor: Corpus for Analyzing LLMs for Etiquettescited by 33

Built on by

Not cited yet.