# Culture-Aware Machine Translation in Large Language Models: Benchmarking and Investigation

arXiv: https://arxiv.org/abs/2604.24361 · HTML: https://mlsanity.com/paper/2604.24361/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 65 references that are in the map.
- First posted on arXiv 27 Apr 2026.

## Builds on (most cited first, 16 of 16)

- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis](https://mlsanity.com/paper/2304.04675/) (2023)
- [GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4](https://mlsanity.com/paper/2310.13988/) (2023)
- [Benchmarking Machine Translation with Cultural Awareness](https://mlsanity.com/paper/2305.14328/) (2023)
- [LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages](https://mlsanity.com/paper/2407.05975/) (2024)
- [Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language Models](https://mlsanity.com/paper/2308.13961/) (2023)
- [GlobeSumm: A Challenging Benchmark Towards Unifying Multi-lingual, Cross-lingual and Multi-document News Summarization](https://mlsanity.com/paper/2410.04087/) (2024)
- [Cultural Adaptation of Menus: A Fine-Grained Approach](https://mlsanity.com/paper/2408.13534/) (2024)
- [Proverbs Run in Pairs: Evaluating Proverb Translation Capability of Large Language Model](https://mlsanity.com/paper/2501.11953/) (2025)
- [Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation](https://mlsanity.com/paper/2504.01919/) (2025)
- [DITING: A Multi-Agent Evaluation Framework for Benchmarking Web Novel Translation](https://mlsanity.com/paper/2510.09116/) (2025)
- [Aligning Translation-Specific Understanding to General Understanding in Large Language Models](https://mlsanity.com/paper/2401.05072/) (2024)
- [LangGPS: Language Separability Guided Data Pre-Selection for Joint Multilingual Instruction Tuning](https://mlsanity.com/paper/2511.10229/) (2025)
- [CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning](https://mlsanity.com/paper/2506.00875/) (2025)

## Built on by (most cited first, 4 of 4)

- [CultureForest: Understanding and Evaluating Cultural Norm Grounded Reasoning in LLMs](https://mlsanity.com/paper/2606.01879/) (2026)
- [Beyond "To whom it may concern": Tailoring Machine Translation to Audience and Intent](https://mlsanity.com/paper/2606.03259/) (2026)
- [Recognized but Not Produced: A Generation Benchmark for Culturally Specific Kinship Terms](https://mlsanity.com/paper/2609.26942/) (2026)
- [CRISP: Cultural Reward Modeling for Implicit Situated Propriety](https://mlsanity.com/paper/2609.34345/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have achieved strong performance in general machine translation, yet their ability in culture-aware scenarios remains poorly understood. To bridge this gap, we introduce CanMT, a Culture-Aware Novel-Driven Parallel Dataset for Machine Translation, together with a theoretically grounded, multi-dimensional evaluation framework for assessing cultural translation quality. Leveraging CanMT, we systematically evaluate a wide range of LLMs and translation systems under different translation strategy constraints. Our findings reveal substantial performance disparities across models and demonstrate that translation strategies exert a systematic influence on model behavior. Further analysis shows that translation difficulty varies across types of culture-specific items, and that a persistent gap remains between models' recognition of culture-specific knowledge and their ability to correctly operationalize it in translation outputs. In addition, incorporating reference translations is shown to substantially improve evaluation reliability in LLM-as-a-judge, underscoring their essential role in assessing culture-aware translation quality. The corpus and code are available at CanMT.