mlsanityData to 14 Oct 2026

Who Brought Easter Eggs to Eid? Auditing LLM-Generated Cultural Translation of Math Word Problems Across Languages and Regions

arXiv 2606.11009 ↗First posted 9 Jun 2026cs.CL cs.CYcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models are increasingly used to adapt math word problems for personalized learning at scale, but it remains an open question whether those adaptations are consistent across models, preserve cultural diversity at scale, and reveal which cultural entities models treat as most salient. We analyze how Claude Opus 4, GPT-4.1, and Gemini 2.5 Pro adapt 60 English math word problems into Bengali, Hindi, Punjabi (India), Urdu, Sindhi (Pakistan), Italian, and Sicilian (Italy), a language set spanning the full resource spectrum, from high-resource Italian and Hindi to under-studied Sindhi, Sicilian, and Punjabi. We annotate 6,489 entity transformations, coding whether models preserve, localize, generalize, omit, or change entities such as names, foods, and places. Models agree on transformation type in 62.5% of cases and on specific substitutions in only 33.5%, meaning model choice directly shapes which cultural world students encounter. All 21 language-model combinations show entropy collapse, with adaptation compressing rather than expanding cultural diversity. Models prioritize surface markers such as names, foods, and currencies while preserving deeper structural features such as grade-level systems that embed culturally specific assumptions. Despite prompts specifying target countries, models misattribute regional context by using Bangladeshi taka for Indian Bengali students and produce cross-cultural contamination, such as adapting egg hunts as Eid activities. Some failures are visible in individual translations. Others, including diversity collapse, systematic preference for surface markers, and consistent regional misattribution, emerge only through corpus-level analysis. The surface plausibility that makes adapted problems look correct is precisely what makes deeper failures easy to overlook.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 36 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

9 of 36 references.

References · 9

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Towards Measuring the Representation of Subjective Global Opinions in Language Modelscited by 437
  3. 2023Assessing Cross-Cultural Alignment between ChatGPT and Human Societies: An Empirical Studycited by 206
  4. 2022Probing Pre-Trained Language Models for Cross-Cultural Differences in Valuescited by 149
  5. 2024Unintended Impacts of LLM Alignment on Global Representationcited by 66
  6. 2024AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuancescited by 60
  7. 2024CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Promptingcited by 55
  8. 2025Lost in Cultural Translation: Do LLMs Struggle with Math Across Cultural Contexts?cited by 9
  9. 2025Mathematics Isn't Culture-Free: Probing Cultural Gaps via Entity and Scenario Perturbationscited by 3

Built on by

Not cited yet.