# Contextual Refinement of Translations: Large Language Models for Sentence and Document-Level Post-Editing

arXiv: https://arxiv.org/abs/2310.14855 · HTML: https://mlsanity.com/paper/2310.14855/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 44 references that are in the map.
- First posted on arXiv 23 Oct 2023.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [Prompting PaLM for Translation: Assessing Strategies and Performance](https://mlsanity.com/paper/2211.09102/) (2022)
- [Context-Aware Neural Machine Translation Learns Anaphora Resolution](https://mlsanity.com/paper/1805.10163/) (2018)
- [Neural Machine Translation with Extended Context](https://mlsanity.com/paper/1708.05943/) (2017)
- [Leveraging GPT-4 for Automatic Translation Post-Editing](https://mlsanity.com/paper/2305.14878/) (2023)
- [DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders](https://mlsanity.com/paper/2106.13736/) (2021)
- [G-Transformer for Document-level Machine Translation](https://mlsanity.com/paper/2105.14761/) (2021)
- [Context-Aware Monolingual Repair for Neural Machine Translation](https://mlsanity.com/paper/1909.01383/) (2019)
- [Does Neural Machine Translation Benefit from Larger Context?](https://mlsanity.com/paper/1704.05135/) (2017)
- [When Does Translation Require Context? A Data-driven, Multilingual Exploration](https://mlsanity.com/paper/2109.07446/) (2021)
- [Escaping the sentence-level paradigm in machine translation](https://mlsanity.com/paper/2304.12959/) (2023)
- [Augmenting Large Language Model Translators via Translation Memories](https://mlsanity.com/paper/2305.17367/) (2023)
- [Pre-Translation for Neural Machine Translation](https://mlsanity.com/paper/1610.05243/) (2016)
- [Improving Long Context Document-Level Machine Translation](https://mlsanity.com/paper/2306.05183/) (2023)
- [Encoding Sentence Position in Context-Aware Neural Machine Translation with Concatenation](https://mlsanity.com/paper/2302.06459/) (2023)

## Built on by (most cited first, 21 of 21)

- [TEaR: Improving LLM-based Machine Translation with Systematic Self-Refinement](https://mlsanity.com/paper/2402.16379/) (2024)
- [Ladder: A Model-Agnostic Framework Boosting LLM-based Machine Translation to the Next Level](https://mlsanity.com/paper/2406.15741/) (2024)
- [Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning](https://mlsanity.com/paper/2410.02631/) (2024)
- [Investigating Length Issues in Document-level Machine Translation](https://mlsanity.com/paper/2412.17592/) (2024)
- [DeMPT: Decoding-enhanced Multi-phase Prompt Tuning for Making LLMs Be Better Context-aware Translators](https://mlsanity.com/paper/2402.15200/) (2024)
- [Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinement](https://mlsanity.com/paper/2504.05614/) (2025)
- [Blending LLMs into Cascaded Speech Translation: KIT's Offline Speech Translation System for IWSLT 2024](https://mlsanity.com/paper/2406.16777/) (2024)
- [KIT's Offline Speech Translation and Instruction Following Submission for IWSLT 2025](https://mlsanity.com/paper/2505.13036/) (2025)
- [Aligning Translation-Specific Understanding to General Understanding in Large Language Models](https://mlsanity.com/paper/2401.05072/) (2024)
- [Language Modelling Approaches to Adaptive Machine Translation](https://mlsanity.com/paper/2401.14559/) (2024)
- [Speech Translation Refinement using Large Language Models](https://mlsanity.com/paper/2501.15090/) (2025)
- [Exploring In-context Example Generation for Machine Translation](https://mlsanity.com/paper/2506.00507/) (2025)
- [GRAFT: A Graph-based Flow-aware Agentic Framework for Document-level Machine Translation](https://mlsanity.com/paper/2507.03311/) (2025)
- [GAIA-v2-LILT: Multilingual Adaptation of Agent Benchmark beyond Translation](https://mlsanity.com/paper/2604.24929/) (2026)
- [Investigating Numerical Translation with Large Language Models](https://mlsanity.com/paper/2501.04927/) (2025)
- [OpenWHO: A Document-Level Parallel Corpus for Health Translation in Low-Resource Languages](https://mlsanity.com/paper/2508.16048/) (2025)
- [STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translation](https://mlsanity.com/paper/2608.27161/) (2026)
- [Interpretable Style Takagi-Sugeno-Kang Fuzzy Clustering](https://mlsanity.com/paper/2504.05125/) (2025)
- [Beyond the Sentence: A Survey on Context-Aware Machine Translation with Large Language Models](https://mlsanity.com/paper/2506.07583/) (2025)
- [LangMark: A Multilingual Dataset for Automatic Post-Editing](https://mlsanity.com/paper/2511.17153/) (2025)
- [Worlds Within Words: Translating Culture in Ancient Chinese Texts with Multi-Agent Coordination](https://mlsanity.com/paper/2606.01276/) (2026)

## Abstract (arXiv)

Large Language Models (LLM's) have demonstrated considerable success in various Natural Language Processing tasks, but they have yet to attain state-of-the-art performance in Neural Machine Translation (NMT). Nevertheless, their significant performance in tasks demanding a broad understanding and contextual processing shows their potential for translation. To exploit these abilities, we investigate using LLM's for MT and explore recent parameter-efficient fine-tuning techniques. Surprisingly, our initial experiments find that fine-tuning for translation purposes even led to performance degradation. To overcome this, we propose an alternative approach: adapting LLM's as Automatic Post-Editors (APE) rather than direct translators. Building on the LLM's exceptional ability to process and generate lengthy sequences, we also propose extending our approach to document-level translation. We show that leveraging Low-Rank-Adapter fine-tuning for APE can yield significant improvements across both sentence and document-level metrics while generalizing to out-of-domain data. Most notably, we achieve a state-of-the-art accuracy rate of 89% on the ContraPro test set, which specifically assesses the model's ability to resolve pronoun ambiguities when translating from English to German. Lastly, we investigate a practical scenario involving manual post-editing for document-level translation, where reference context is made available. Here, we demonstrate that leveraging human corrections can significantly reduce the number of edits required for subsequent translations (Interactive Demo for integrating manual feedback can be found here: https://huggingface.co/spaces/skoneru/contextualrefinementende).