# Guiding Large Language Models to Post-Edit Machine Translation with Error Annotations

arXiv: https://arxiv.org/abs/2404.07851 · HTML: https://mlsanity.com/paper/2404.07851/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 48 references that are in the map.
- First posted on arXiv 11 Apr 2024.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [BLEURT: Learning Robust Metrics for Text Generation](https://mlsanity.com/paper/2004.04696/) (2020)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translation](https://mlsanity.com/paper/2104.14478/) (2021)
- [MEGA: Multilingual Evaluation of Generative AI](https://mlsanity.com/paper/2303.12528/) (2023)
- [Few-shot Learning with Multilingual Language Models](https://mlsanity.com/paper/2112.10668/) (2021)
- [Generating Sequences by Learning to Self-Correct](https://mlsanity.com/paper/2211.00053/) (2022)
- [xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection](https://mlsanity.com/paper/2310.10482/) (2023)
- [Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies](https://mlsanity.com/paper/2308.03188/) (2023)
- [Tower: An Open Multilingual Large Language Model for Translation-Related Tasks](https://mlsanity.com/paper/2402.17733/) (2024)
- [Levenshtein Transformer](https://mlsanity.com/paper/1905.11006/) (2019)
- [Is ChatGPT A Good Translator? Yes With GPT-4 As The Engine](https://mlsanity.com/paper/2301.08745/) (2023)
- [GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4](https://mlsanity.com/paper/2310.13988/) (2023)
- [The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation](https://mlsanity.com/paper/2308.07286/) (2023)
- [CoEdIT: Text Editing by Task-Specific Instruction Tuning](https://mlsanity.com/paper/2305.09857/) (2023)
- [Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOM](https://mlsanity.com/paper/2303.01911/) (2023)
- [Leveraging GPT-4 for Automatic Translation Post-Editing](https://mlsanity.com/paper/2305.14878/) (2023)
- [LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback](https://mlsanity.com/paper/2311.09336/) (2023)
- [DEMETR: Diagnosing Evaluation Metrics for Translation](https://mlsanity.com/paper/2210.13746/) (2022)
- [Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing](https://mlsanity.com/paper/1605.04800/) (2016)
- [Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decoding](https://mlsanity.com/paper/2311.02851/) (2023)

## Built on by (most cited first, 22 of 22)

- [When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs](https://mlsanity.com/paper/2406.01297/) (2024)
- [X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scale](https://mlsanity.com/paper/2410.03115/) (2024)
- [MQM-APE: Toward High-Quality Error Annotation Predictors with Automatic Post-Editing in LLM Translation Evaluators](https://mlsanity.com/paper/2409.14335/) (2024)
- [xTower: A Multilingual LLM for Explaining and Correcting Translation Errors](https://mlsanity.com/paper/2406.19482/) (2024)
- [SciDoc2Diagrammer-MAF: Towards Generation of Scientific Diagrams from Documents guided by Multi-Aspect Feedback Refinement](https://mlsanity.com/paper/2409.19242/) (2024)
- [Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation](https://mlsanity.com/paper/2503.04554/) (2025)
- [Lost in Literalism: How Supervised Training Shapes Translationese in LLMs](https://mlsanity.com/paper/2503.04369/) (2025)
- [MAPS: A Multilingual Benchmark for Agent Performance and Security](https://mlsanity.com/paper/2505.15935/) (2025)
- [An Interdisciplinary Approach to Human-Centered Machine Translation](https://mlsanity.com/paper/2506.13468/) (2025)
- [Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologies](https://mlsanity.com/paper/2410.08860/) (2024)
- [Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinement](https://mlsanity.com/paper/2504.05614/) (2025)
- [Automatic Input Rewriting Improves Translation with Large Language Models](https://mlsanity.com/paper/2502.16682/) (2025)
- [LaSEr-Edit: Localized Span-level Error Editing with Energy-based Localization](https://mlsanity.com/paper/2407.00740/) (2024)
- [Speech Translation Refinement using Large Language Models](https://mlsanity.com/paper/2501.15090/) (2025)
- [Translation Analytics for Freelancers: I. Introduction, Data Preparation, Baseline Evaluations](https://mlsanity.com/paper/2504.14619/) (2025)
- [Translation in the Wild](https://mlsanity.com/paper/2505.23548/) (2025)
- [Investigating Numerical Translation with Large Language Models](https://mlsanity.com/paper/2501.04927/) (2025)
- [TULUN: Transparent and Adaptable Low-resource Machine Translation](https://mlsanity.com/paper/2505.18683/) (2025)
- [LangMark: A Multilingual Dataset for Automatic Post-Editing](https://mlsanity.com/paper/2511.17153/) (2025)
- [PAMT: Process-Aligned Reinforcement Learning for Multi-Domain Machine Translation](https://mlsanity.com/paper/2608.03077/) (2026)
- [North Small Translate: Advanced Cost-Effective Translation (Cohere CAT+)](https://mlsanity.com/paper/2609.13916/) (2026)
- [Confident but Wrong: A Constrained Decoding Diagnostic for Low-Resource Automatic Post-Editing](https://mlsanity.com/paper/2609.29680/) (2026)

## Abstract (arXiv)

Machine Translation (MT) remains one of the last NLP tasks where large language models (LLMs) have not yet replaced dedicated supervised systems. This work exploits the complementary strengths of LLMs and supervised MT by guiding LLMs to automatically post-edit MT with external feedback on its quality, derived from Multidimensional Quality Metric (MQM) annotations. Working with LLaMA-2 models, we consider prompting strategies varying the nature of feedback provided and then fine-tune the LLM to improve its ability to exploit the provided guidance. Through experiments on Chinese-English, English-German, and English-Russian MQM data, we demonstrate that prompting LLMs to post-edit MT improves TER, BLEU and COMET scores, although the benefits of fine-grained feedback are not clear. Fine-tuning helps integrate fine-grained feedback more effectively and further improves translation quality based on both automatic and human evaluation.