mlsanityData to 14 Oct 2026

Benchmarking Machine Translation with Cultural Awareness

arXiv 2305.14328 ↗First posted 23 May 2023cs.CLcited by 53

Not on the map. Why?

Abstract (arXiv)

Translating culture-related content is vital for effective cross-cultural communication. However, many culture-specific items (CSIs) often lack viable translations across languages, making it challenging to collect high-quality, diverse parallel corpora with CSI annotations. This difficulty hinders the analysis of cultural awareness of machine translation (MT) systems, including traditional neural MT and the emerging MT paradigm using large language models (LLM). To address this gap, we introduce a novel parallel corpus, enriched with CSI annotations in 6 language pairs for investigating Culturally-Aware Machine Translation--CAMT. Furthermore, we design two evaluation metrics to assess CSI translations, focusing on their pragmatic translation quality. Our findings show the superior ability of LLMs over neural MTs in leveraging external cultural knowledge for translating CSIs, especially those lacking translations in the target culture.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). Builds on 24 of its 46 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

24 of 46 references.

References · 24

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  4. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  5. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  6. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  7. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
  8. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  9. 2019Massively Multilingual Neural Machine Translationcited by 246
  10. 2022Challenges and Strategies in Cross-Cultural NLPcited by 218
  11. 2020Nearest Neighbor Machine Translationcited by 209
  12. 2022On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Modelcited by 81
12 more
  1. 2023Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translationcited by 74
  2. 2019Training Neural Machine Translation To Apply Terminology Constraintscited by 73
  3. 2016Incorporating Discrete Translation Lexicons into Neural Machine Translationcited by 33
  4. 2019Domain Adaptation of Neural Machine Translation by Lexicon Inductioncited by 33
  5. 2022Using natural language prompts for machine translationcited by 27
  6. 2022FRMT: A Benchmark for Few-Shot Region-Aware Machine Translationcited by 18
  7. 2021On the Evaluation of Machine Translation for Terminology Consistencycited by 15
  8. 2021DEEP: DEnoising Entity Pre-training for Neural Machine Translationcited by 15
  9. 2017SLING: A framework for frame semantic parsingcited by 12
  10. 2023varepsilon K'U <MASK>: Integrating Yor`ub'a cultural greetings into machine translationcited by 8
  11. 2020The ApposCorpus: A new multilingual, multi-domain dataset for factual appositive generationcited by 6
  12. 2017Translating Phrases in Neural Machine Translationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 8; 2025: 19; 2026: 23.

53 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Uglycited by 387
  2. 2024Survey of Cultural Awareness in Language Models: Text and Beyondcited by 103
  3. 2025Culture is Not Trivia: Sociocultural Theory for Cultural NLPcited by 40
  4. 2023Are Multilingual LLMs Culturally-Diverse Reasoners? An Investigation into Multicultural Proverbs and Sayingscited by 32
  5. 2024Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Artcited by 26
  6. 2025Learn Globally, Speak Locally: Bridging the Gaps in Multilingual Reasoningcited by 24
  7. 2025R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learningcited by 21
  8. 2024Multilingual Large Language Models: A Systematic Surveycited by 18
  9. 2025Charting the Landscape of African NLP: Mapping Progress and Shaping the Road Aheadcited by 16
  10. 2024Designing Efficient LLM Accelerators for Edge Devicescited by 13
  11. 2025MAKIEval: A Multilingual Automatic WiKidata-based Framework for Cultural Awareness Evaluation for LLMscited by 13
  12. 2025Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translationcited by 8
38 more
  1. 2023Bridging Background Knowledge Gaps in Translation with Automatic Explicitationcited by 7
  2. 2024What Is Missing in Multilingual Visual Reasoning and How to Fix Itcited by 7
  3. 2025An Interdisciplinary Approach to Human-Centered Machine Translationcited by 6
  4. 2026Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluationcited by 6
  5. 2025DIWALI: Diversity and Inclusivity aWare cuLture specific Items for India: Dataset and Assessment of LLMs for Cultural Text Adaptation in Indian Contextcited by 5
  6. 2025CaMMT: Benchmarking Culturally Aware Multimodal Machine Translationcited by 4
  7. 2025Towards Style Alignment in Cross-Cultural Translationcited by 4
  8. 2025UPDESH: Synthesizing Grounded Instruction Tuning Data for 13 Indic Languagescited by 4
  9. 2026Culture-Aware Machine Translation in Large Language Models: Benchmarking and Investigationcited by 4
  10. 2025Exploring the Translation Mechanism of Large Language Modelscited by 3
  11. 2025Fine-Tuning LLMs for Low-Resource Dialect Translation: The Case of Lebanesecited by 3
  12. 2026NeoAMT: Neologism-Aware Agentic Machine Translation with Reinforcement Learningcited by 3
  13. 2026LQM: Linguistically Motivated Multidimensional Quality Metrics for Machine Translationcited by 3
  14. 2024Vernacular? I Barely Know Her: Challenges with Style Control and Stereotypingcited by 2
  15. 2025Building Large-Scale English-Romanian Literary Translation Resources with Open Modelscited by 2
  16. 2026Benchmarking Machine Translation on Chinese Social Media Textscited by 2
  17. 2026Round-Trip Translation Reveals What Frontier Multilingual Benchmarks Misscited by 2
  18. 2026Beyond Literal Translation: Evaluating Cultural Effectiveness in Social Media UGCcited by 2
  19. 2026The Culture Funnel: You Can't Align What isn't in the Datacited by 2
  20. 2024Exploiting LLMs' Reasoning Capability to Infer Implicit Concepts in Legal Information Retrievalcited by 1
  21. 2025HypoEval: Hypothesis-Guided Evaluation for Natural Language Generationcited by 1
  22. 2025Liaozhai through the Looking-Glass: On Paratextual Explicitation of Culture-Bound Terms in Machine Translationcited by 1
  23. 2026Incentivizing Parametric Knowledge via Reinforcement Learning with Verifiable Rewards for Cross-Cultural Entity Translationcited by 1
  24. 2026The GaoYao Benchmark: A Comprehensive Framework for Evaluating Multilingual and Multicultural Abilities of Large Language Modelscited by 1
  25. 2026Defining Cultural Capabilities for AI Evaluation: A Taxonomy Grounded in Intercultural Communication Theorycited by 1
  26. 2026Last Translation Benchmarkcited by 1
  27. 2024Enhancing Entertainment Translation for Indian Languages using Adaptive Context, Style and LLMscited by 0
  28. 2025Toward Machine Interpreting: Lessons from Human Interpreting Studiescited by 0
  29. 2025MedCOD: Enhancing English-to-Spanish Medical Translation of Large Language Models Using Enriched Chain-of-Dictionary Frameworkcited by 0
  30. 2025Do LLMs Understand Wine Descriptors Across Cultures? A Benchmark for Cultural Adaptations of Wine Reviewscited by 0
  31. 2026Who Benchmarks the Benchmarks? A Case Study of LLM Evaluation in Icelandiccited by 0
  32. 2026Discovering Lexical Gaps Using Embeddings from Multilingual LLMscited by 0
  33. 2026Cultural Fidelity in English-to-Hindi Translation: A Preservation-Fluency Frontier for Gender Recoverabilitycited by 0
  34. 2026Worlds Within Words: Translating Culture in Ancient Chinese Texts with Multi-Agent Coordinationcited by 0
  35. 2026MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languagescited by 0
  36. 2026On the Systematic Challenges of Culturally Loaded Machine Translation: Dream of the Red Chamber as the Cultural Lenscited by 0
  37. 2026Translation with Thought: Difficulty-Adaptive Reasoning via Reinforcement Learning for Multi-Domain Machine Translationcited by 0
  38. 2026PAMT: Process-Aligned Reinforcement Learning for Multi-Domain Machine Translationcited by 0

50 most cited of 53 All 53 (JSON)