mlsanityData to 14 Oct 2026

Guiding Large Language Models to Post-Edit Machine Translation with Error Annotations

arXiv 2404.07851 ↗First posted 11 Apr 2024cs.CL cs.AIcited by 22

Not on the map. Why?

Abstract (arXiv)

Machine Translation (MT) remains one of the last NLP tasks where large language models (LLMs) have not yet replaced dedicated supervised systems. This work exploits the complementary strengths of LLMs and supervised MT by guiding LLMs to automatically post-edit MT with external feedback on its quality, derived from Multidimensional Quality Metric (MQM) annotations. Working with LLaMA-2 models, we consider prompting strategies varying the nature of feedback provided and then fine-tune the LLM to improve its ability to exploit the provided guidance. Through experiments on Chinese-English, English-German, and English-Russian MQM data, we demonstrate that prompting LLMs to post-edit MT improves TER, BLEU and COMET scores, although the benefits of fine-grained feedback are not clear. Fine-tuning helps integrate fine-grained feedback more effectively and further improves translation quality based on both automatic and human evaluation.

Key facts. Cited by 22 papers in the map (data through 14 Oct 2026). Builds on 31 of its 48 references that are in the map. First posted on arXiv 11 Apr 2024.

Builds on

31 of 48 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  7. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  9. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  10. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  11. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
  12. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
19 more
  1. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  2. 2023MEGA: Multilingual Evaluation of Generative AIcited by 293
  3. 2021Few-shot Learning with Multilingual Language Modelscited by 278
  4. 2022Generating Sequences by Learning to Self-Correctcited by 277
  5. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
  6. 2023Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategiescited by 238
  7. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  8. 2019Levenshtein Transformercited by 199
  9. 2023Is ChatGPT A Good Translator? Yes With GPT-4 As The Enginecited by 176
  10. 2023GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4cited by 146
  11. 2023The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluationcited by 95
  12. 2023CoEdIT: Text Editing by Task-Specific Instruction Tuningcited by 68
  13. 2023Investigating the Translation Performance of a Large Multilingual Language Model: the Case of BLOOMcited by 59
  14. 2023Leveraging GPT-4 for Automatic Translation Post-Editingcited by 56
  15. 2023LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedbackcited by 39
  16. 2022DEMETR: Diagnosing Evaluation Metrics for Translationcited by 36
  17. 2016Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editingcited by 13
  18. 2023Improving Machine Translation with Large Language Models: A Preliminary Study with Cooperative Decodingcited by 10
  19. 2020Incorporating Terminology Constraints in Automatic Post-Editingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 7; 2025: 12; 2026: 3.

22 papers.

Sort:

By area:

  1. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  2. 2024X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scalecited by 36
  3. 2024MQM-APE: Toward High-Quality Error Annotation Predictors with Automatic Post-Editing in LLM Translation Evaluatorscited by 22
  4. 2024xTower: A Multilingual LLM for Explaining and Correcting Translation Errorscited by 17
  5. 2024SciDoc2Diagrammer-MAF: Towards Generation of Scientific Diagrams from Documents guided by Multi-Aspect Feedback Refinementcited by 15
  6. 2025Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translationcited by 14
  7. 2025Lost in Literalism: How Supervised Training Shapes Translationese in LLMscited by 11
  8. 2025MAPS: A Multilingual Benchmark for Agent Performance and Securitycited by 7
  9. 2025An Interdisciplinary Approach to Human-Centered Machine Translationcited by 6
  10. 2024Audio Description Generation in the Era of LLMs and VLMs: A Review of Transferable Generative AI Technologiescited by 5
  11. 2025Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinementcited by 5
  12. 2025Automatic Input Rewriting Improves Translation with Large Language Modelscited by 4
10 more
  1. 2024LaSEr-Edit: Localized Span-level Error Editing with Energy-based Localizationcited by 2
  2. 2025Speech Translation Refinement using Large Language Modelscited by 2
  3. 2025Translation Analytics for Freelancers: I. Introduction, Data Preparation, Baseline Evaluationscited by 2
  4. 2025Translation in the Wildcited by 2
  5. 2025Investigating Numerical Translation with Large Language Modelscited by 1
  6. 2025TULUN: Transparent and Adaptable Low-resource Machine Translationcited by 0
  7. 2025LangMark: A Multilingual Dataset for Automatic Post-Editingcited by 0
  8. 2026PAMT: Process-Aligned Reinforcement Learning for Multi-Domain Machine Translationcited by 0
  9. 2026North Small Translate: Advanced Cost-Effective Translation (Cohere CAT+)cited by 0
  10. 2026Confident but Wrong: A Constrained Decoding Diagnostic for Low-Resource Automatic Post-Editingcited by 0