The Role of Prompt Language and Translation-Theory-Driven Prompts in Large Language Models: A Case Study on Spanish-Chinese Journalistic Translation
Not on the map. Why?
Abstract (arXiv)
This study examines how prompt language and translation theory-driven prompt design influence the quality of Spanish-Chinese journalistic translations generated by GPT-5.2. A parallel corpus of four editorials from El Pais was translated under 48 experimental conditions (4 prompt types, 3 prompt languages, and 4 articles). Translation quality was assessed using BLEU and BERTScore-F1 for automated evaluation, alongside human evaluation based on the Multidimensional Quality Metrics (MQM) framework. Automated metrics identified the baseline prompt (BASE) as the best-performing condition, whereas human evaluation ranked the brief-oriented prompt (BRIEF) highest (MQM: 8.66 vs. 7.84), a reversal likely attributable to the single-reference constraint inherent in automated measures. Sub-error type analysis revealed that translation theory-driven prompts selectively reduced Awkward style errors, while Unidiomatic style errors persisted across conditions. Prompt language had a negligible impact under both evaluation paradigms. These results indicate that translation theory-driven prompts can yield measurable quality gains under expert evaluation of journalistic translations, although their pedagogical implications for language learners remain suggestive and require validation through user-based studies.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 62 references that are in the map. First posted on arXiv 3 Jul 2026.
Builds on
20 of 62 references.
References · 20
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
- 2022Language Models are Multilingual Chain-of-Thought Reasonerscited by 518
- 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
- 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
- 2023MEGA: Multilingual Evaluation of Generative AIcited by 291
- 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 247
- 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 172
- 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 156
- 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 144
- 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
8 more
- 2024Should We Respect LLMs? A Cross-Lingual Study on the Influence of Prompt Politeness on LLM Performancecited by 41
- 2023How to Design Translation Prompts for ChatGPT: An Empirical Studycited by 31
- 2024Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Modelscited by 31
- 2025Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMscited by 22
- 2025The Token Tax: Systematic Bias in Multilingual Tokenizationcited by 13
- 2024Prompting ChatGPT for Translation: A Comparative Analysis of Translation Brief and Persona Promptscited by 12
- 2023Linguistically Informed ChatGPT Prompts to Enhance Japanese-Chinese Machine Translation: A Case Study on Attributive Clausescited by 8
- 2025How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Studycited by 4
Built on by
Not cited yet.