# Leveraging GPT-4 for Automatic Translation Post-Editing

arXiv: https://arxiv.org/abs/2305.14878 · HTML: https://mlsanity.com/paper/2305.14878/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 51 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 30 of 36)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://mlsanity.com/paper/2305.04388/) (2023)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Language Models are Multilingual Chain-of-Thought Reasoners](https://mlsanity.com/paper/2210.03057/) (2022)
- [Are Emergent Abilities of Large Language Models a Mirage?](https://mlsanity.com/paper/2304.15004/) (2023)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation](https://mlsanity.com/paper/2104.14478/) (2021)
- [The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning](https://mlsanity.com/paper/2205.03401/) (2022)
- [The Curious Case of Hallucinations in Neural Machine Translation](https://mlsanity.com/paper/2104.06683/) (2021)
- [Prompting PaLM for Translation: Assessing Strategies and Performance](https://mlsanity.com/paper/2211.09102/) (2022)
- [The unreasonable effectiveness of few-shot learning for machine translation](https://mlsanity.com/paper/2302.01398/) (2023)
- [Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models](https://mlsanity.com/paper/2303.13809/) (2023)
- [Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation](https://mlsanity.com/paper/2208.05309/) (2022)
- [Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection](https://mlsanity.com/paper/2301.07779/) (2023)
- [Context-Aware Monolingual Repair for Neural Machine Translation](https://mlsanity.com/paper/1909.01383/) (2019)
- [Automatic Testing and Improvement of Machine Translation](https://mlsanity.com/paper/1910.02688/) (2019)
- [How to Design Translation Prompts for ChatGPT: An Empirical Study](https://mlsanity.com/paper/2304.02182/) (2023)
- [Structure-Invariant Testing for Machine Translation](https://mlsanity.com/paper/1907.08710/) (2019)
- [SALTED: A Framework for SAlient Long-Tail Translation Error Detection](https://mlsanity.com/paper/2205.09988/) (2022)
- [Testing Machine Translation via Referential Transparency](https://mlsanity.com/paper/2004.10361/) (2020)
- [Can Automatic Post-Editing Improve NMT?](https://mlsanity.com/paper/2009.14395/) (2020)
- [Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translation](https://mlsanity.com/paper/2212.09631/) (2022)
- [Neural Machine Translation Quality and Post-Editing Performance](https://mlsanity.com/paper/2109.05016/) (2021)

## Built on by (most cited first, 30 of 56)

- [When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs](https://mlsanity.com/paper/2406.01297/) (2024)
- [Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies](https://mlsanity.com/paper/2308.03188/) (2023)
- [Tower: An Open Multilingual Large Language Model for Translation-Related Tasks](https://mlsanity.com/paper/2402.17733/) (2024)
- [Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers](https://mlsanity.com/paper/2404.04925/) (2024)
- [Large Language Models have Intrinsic Self-Correction Ability](https://mlsanity.com/paper/2406.15673/) (2024)
- [TEaR: Improving LLM-based Machine Translation with Systematic Self-Refinement](https://mlsanity.com/paper/2402.16379/) (2024)
- [X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale](https://mlsanity.com/paper/2410.03115/) (2024)
- [Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AI](https://mlsanity.com/paper/2309.12444/) (2023)
- [Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning](https://mlsanity.com/paper/2310.00647/) (2023)
- [G-DIG: Towards Gradient-based Diverse and High-quality Instruction Data Selection for Machine Translation](https://mlsanity.com/paper/2405.12915/) (2024)
- [Improving Translation Faithfulness of Large Language Models via Augmenting Instructions](https://mlsanity.com/paper/2308.12674/) (2023)
- [Iterative Translation Refinement with Large Language Models](https://mlsanity.com/paper/2306.03856/) (2023)
- [Learning to Refine with Fine-Grained Natural Language Feedback](https://mlsanity.com/paper/2407.02397/) (2024)
- [Guiding Large Language Models to Post-Edit Machine Translation with Error Annotations](https://mlsanity.com/paper/2404.07851/) (2024)
- [MQM-APE: Toward High-Quality Error Annotation Predictors with Automatic Post-Editing in LLM Translation Evaluators](https://mlsanity.com/paper/2409.14335/) (2024)
- [Contextual Refinement of Translations: Large Language Models for Sentence and Document-Level Post-Editing](https://mlsanity.com/paper/2310.14855/) (2023)
- [Mitigating the Language Mismatch and Repetition Issues in LLM-based Machine Translation via Model Editing](https://mlsanity.com/paper/2410.07054/) (2024)
- [Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice?](https://mlsanity.com/paper/2404.14122/) (2024)
- [xTower: A Multilingual LLM for Explaining and Correcting Translation Errors](https://mlsanity.com/paper/2406.19482/) (2024)
- [An Empirical Study of Translation Hypothesis Ensembling with Large Language Models](https://mlsanity.com/paper/2310.11430/) (2023)
- [Ladder: A Model-Agnostic Framework Boosting LLM-based Machine Translation to the Next Level](https://mlsanity.com/paper/2406.15741/) (2024)
- [Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning](https://mlsanity.com/paper/2410.02631/) (2024)
- [mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humans](https://mlsanity.com/paper/2406.04215/) (2024)
- [Creative and Context-Aware Translation of East Asian Idioms with GPT-4](https://mlsanity.com/paper/2410.00988/) (2024)
- [MAMM-Refine: A Recipe for Improving Faithfulness in Generation with Multi-Agent Collaboration](https://mlsanity.com/paper/2503.15272/) (2025)
- [Semantic and Expressive Variation in Image Captions Across Languages](https://mlsanity.com/paper/2310.14356/) (2023)
- [Multilingual Prompt Engineering in Large Language Models: A Survey Across NLP Tasks](https://mlsanity.com/paper/2505.11665/) (2025)
- [ECoh: Turn-level Coherence Evaluation for Multilingual Dialogues](https://mlsanity.com/paper/2407.11660/) (2024)
- [Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinement](https://mlsanity.com/paper/2504.05614/) (2025)
- [Will It Still Be True Tomorrow? Multilingual Evergreen Question Classification to Improve Trustworthy QA](https://mlsanity.com/paper/2505.21115/) (2025)

## Abstract (arXiv)

While Neural Machine Translation (NMT) represents the leading approach to Machine Translation (MT), the outputs of NMT models still require translation post-editing to rectify errors and enhance quality under critical settings. In this work, we formalize the task of direct translation post-editing with Large Language Models (LLMs) and explore the use of GPT-4 to automatically post-edit NMT outputs across several language pairs. Our results demonstrate that GPT-4 is adept at translation post-editing, producing meaningful and trustworthy edits to translations that help improve its general quality as well as remove different classes of major errors in translations. In particular, human evaluations on assessing edit trustworthiness show that GPT-4 exhibits a large improvement over the prior state-of-the-art LLM. Notably, we improve upon state-of-the-art performance on WMT-22 English-Chinese, English-German, Chinese-English and German-English language pairs using GPT-4 based post-editing, as evaluated by state-of-the-art MT quality metrics. However, we also show that GPT-4 could produce hallucinated edits, thereby urging caution in its use as an expert translation post-editor.