mlsanityData to 14 Oct 2026

Do GPTs Produce Less Literal Translations?

arXiv 2305.16806 ↗First posted 26 May 2023cs.CL cs.AIcited by 30

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) such as GPT-3 have emerged as general-purpose language models capable of addressing many natural language generation or understanding tasks. On the task of Machine Translation (MT), multiple works have investigated few-shot prompting mechanisms to elicit better translations from LLMs. However, there has been relatively little investigation on how such translations differ qualitatively from the translations generated by standard Neural Machine Translation (NMT) models. In this work, we investigate these differences in terms of the literalness of translations produced by the two systems. Using literalness measures involving word alignment and monotonicity, we find that translations out of English (E-X) from GPTs tend to be less literal, while exhibiting similar or better scores on MT quality metrics. We demonstrate that this finding is borne out in human evaluations as well. We then show that these differences are especially pronounced when translating sentences that contain idiomatic expressions.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 18 of its 37 references that are in the map. First posted on arXiv 26 May 2023.

Builds on

18 of 37 references.

References · 18

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  5. 2016Bag of Tricks for Efficient Text Classificationcited by 876
  6. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  7. 2022News Summarization and Evaluation in the Era of GPT-3cited by 327
  8. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 211
  9. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
  10. 2022Prompting PaLM for Translation: Assessing Strategies and Performancecited by 145
  11. 2020PhraseCut: Language-based Image Segmentation in the Wildcited by 116
  12. 2023Searching for Needles in a Haystack: On the Role of Incidental Bilingualism in PaLM's Translation Capabilitycited by 62
6 more
  1. 2021The paradox of the compositionality of natural language: a neural machine translation case studycited by 59
  2. 2021Facebook AI WMT21 News Translation Task Submissioncited by 57
  3. 2022On the Blind Spots of Model-Based Evaluation Metrics for Text Generationcited by 49
  4. 2022Can Transformer be Too Compositional? Analysing Idiom Processing in Neural Machine Translationcited by 28
  5. 2020EPIE Dataset: A Corpus For Possible Idiomatic Expressionscited by 24
  6. 2022SALTED: A Framework for SAlient Long-Tail Translation Error Detectioncited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 9; 2025: 8; 2026: 5.

30 papers.

Sort:

  1. 2023Speech Translation with Large Language Models: An Industrial Practicecited by 28
  2. 2023Iterative Translation Refinement with Large Language Modelscited by 27
  3. 2024Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?cited by 23
  4. 2024The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilitiescited by 22
  5. 2025Lost in Literalism: How Supervised Training Shapes Translationese in LLMscited by 11
  6. 2023Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decodingcited by 10
  7. 2024To Diverge or Not to Diverge: A Morphosyntactic Perspective on Machine Translation vs Human Translationcited by 10
  8. 2024Creative and Context-Aware Translation of East Asian Idioms with GPT-4cited by 10
  9. 2023Towards Effective Disambiguation for Machine Translation with Large Language Modelscited by 8
  10. 2023SCALE: Synergized Collaboration of Asymmetric Language Translation Enginescited by 3
  11. 2023That was the last straw, we need more: Are Translation Systems Sensitive to Disambiguating Context?cited by 3
  12. 2024Aligning Translation-Specific Understanding to General Understanding in Large Language Modelscited by 3
18 more
  1. 2025Extending CREAMT: Leveraging Large Language Models for Literary Translation Post-Editingcited by 3
  2. 2026Preferences for Idiomatic Language are Acquired Slowly -- and Forgotten Quickly: A Case Study on Swedishcited by 3
  3. 2023Dissecting In-Context Learning of Translations in GPTscited by 2
  4. 2025Exploring In-context Example Generation for Machine Translationcited by 2
  5. 2024Killing Two Flies with One Stone: An Attempt to Break LLMs Using English->Icelandic Idioms and Proper Namescited by 1
  6. 2025Graph-Assisted Culturally Adaptable Idiomatic Translation for Indic Languagescited by 1
  7. 2023Distinguishing Translations by Human, NMT, and ChatGPT: A Linguistic and Statistical Approachcited by 0
  8. 2024Prediction of Translation Techniques for the Translation Processcited by 0
  9. 2024Guiding In-Context Learning of LLMs through Quality Estimation for Machine Translationcited by 0
  10. 2024Large Language Models for Persian leftrightarrow English Idiom Translationcited by 0
  11. 2025Missing the human touch? A computational stylometry analysis of GPT-4 translations of online Chinese literaturecited by 0
  12. 2025ML2B: Benchmarking LLMs on Cross-Lingual ML Pipeline Generationcited by 0
  13. 2025Unlocking Latent Discourse Translation in LLMs Through Quality-Aware Decodingcited by 0
  14. 2025LLMs for Low-Resource Dialect Translation Using Context-Aware Prompting: A Case Study on Sylheticited by 0
  15. 2026A Dataset for Probing Translationese Preferences in English-to-Swedish Translationcited by 0
  16. 2026LLMs Should Incorporate Explicit Mechanisms for Human Empathycited by 0
  17. 2026Testing the Deliteralization Hypothesis in Human and Machine Translationcited by 0
  18. 2026Emotion Profiling in LLM-Based Literary Translation: Systematic Shifts Across MT and Post-Editingcited by 0