# Document-Level Machine Translation with Large Language Models

arXiv: https://arxiv.org/abs/2304.02210 · HTML: https://mlsanity.com/paper/2304.02210/

## Key facts

- Cited by 90 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 56 references that are in the map.
- First posted on arXiv 5 Apr 2023.

## Builds on (most cited first, 30 of 30)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [With Little Power Comes Great Responsibility](https://mlsanity.com/paper/2010.06595/) (2020)
- [Context-Aware Neural Machine Translation Learns Anaphora Resolution](https://mlsanity.com/paper/1805.10163/) (2018)
- [Evaluating Discourse Phenomena in Neural Machine Translation](https://mlsanity.com/paper/1711.00513/) (2017)
- [Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translation](https://mlsanity.com/paper/2302.07856/) (2023)
- [Exploiting Cross-Sentence Context for Neural Machine Translation](https://mlsanity.com/paper/1704.04347/) (2017)
- [G-Transformer for Document-level Machine Translation](https://mlsanity.com/paper/2105.14761/) (2021)
- [Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature](https://mlsanity.com/paper/2210.14250/) (2022)
- [Context-Aware Monolingual Repair for Neural Machine Translation](https://mlsanity.com/paper/1909.01383/) (2019)
- [Learning to Remember Translation History with a Continuous Cache](https://mlsanity.com/paper/1711.09367/) (2017)
- [Incorporating BERT into Parallel Sequence Decoding with Adapters](https://mlsanity.com/paper/2010.06138/) (2020)
- [Rethinking Document-level Neural Machine Translation](https://mlsanity.com/paper/2010.08961/) (2020)
- [TranSmart: A Practical Interactive Machine Translation System](https://mlsanity.com/paper/2105.13072/) (2021)
- [Translating Pro-Drop Languages with Reconstruction Models](https://mlsanity.com/paper/1801.03257/) (2018)
- [One Model to Learn Both: Zero Pronoun Prediction and Translation](https://mlsanity.com/paper/1909.00369/) (2019)
- [A Novel Approach to Dropped Pronoun Translation](https://mlsanity.com/paper/1604.06285/) (2016)
- [Using Whole Document Context in Neural Machine Translation](https://mlsanity.com/paper/1910.07481/) (2019)
- [Towards Making the Most of Context in Neural Machine Translation](https://mlsanity.com/paper/2002.07982/) (2020)
- [Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents](https://mlsanity.com/paper/2109.10341/) (2021)
- [Learning to Jointly Translate and Predict Dropped Pronouns with a Shared Reconstruction Mechanism](https://mlsanity.com/paper/1810.06195/) (2018)

## Built on by (most cited first, 30 of 90)

- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration](https://mlsanity.com/paper/2306.09093/) (2023)
- [A Survey of Large Language Models for Healthcare: from Data, Technology, and Applications to Accountability and Ethics](https://mlsanity.com/paper/2310.05694/) (2023)
- [Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey](https://mlsanity.com/paper/2305.18703/) (2023)
- [Large Language Models Meet NLP: A Survey](https://mlsanity.com/paper/2405.12819/) (2024)
- [AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progress](https://mlsanity.com/paper/2511.08325/) (2025)
- [MarsCode Agent: AI-native Automated Bug Fixing](https://mlsanity.com/paper/2409.00899/) (2024)
- [BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languages](https://mlsanity.com/paper/2305.18098/) (2023)
- [Adapting Large Language Models for Document-Level Machine Translation](https://mlsanity.com/paper/2401.06468/) (2024)
- [TrustGPT: A Benchmark for Trustworthy and Responsible Large Language Models](https://mlsanity.com/paper/2306.11507/) (2023)
- [KnowledGPT: Enhancing Large Language Models with Retrieval and Storage Access on Knowledge Bases](https://mlsanity.com/paper/2308.11761/) (2023)
- [(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts](https://mlsanity.com/paper/2405.11804/) (2024)
- [Parameter-Efficient Fine-Tuning in Large Models: A Survey of Methodologies](https://mlsanity.com/paper/2410.19878/) (2024)
- [Exploring Human-Like Translation Strategy with Large Language Models](https://mlsanity.com/paper/2305.04118/) (2023)
- [What's the Magic Word? A Control Theory of LLM Prompting](https://mlsanity.com/paper/2310.04444/) (2023)
- [Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Media](https://mlsanity.com/paper/2412.18148/) (2024)
- [A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time](https://mlsanity.com/paper/2309.07418/) (2023)
- [Speech Translation with Large Language Models: An Industrial Practice](https://mlsanity.com/paper/2312.13585/) (2023)
- [Escaping the sentence-level paradigm in machine translation](https://mlsanity.com/paper/2304.12959/) (2023)
- [DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory](https://mlsanity.com/paper/2410.08143/) (2024)
- [Beyond Static Datasets: A Deep Interaction Approach to LLM Evaluation](https://mlsanity.com/paper/2309.04369/) (2023)
- [Reassessing Layer Pruning in LLMs: New Insights and Methods](https://mlsanity.com/paper/2411.15558/) (2024)
- [ChatGPT MT: Competitive for High- (but not Low-) Resource Languages](https://mlsanity.com/paper/2309.07423/) (2023)
- [A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debates](https://mlsanity.com/paper/2401.03910/) (2024)
- [The Price of Prompting: Profiling Energy Use in Large Language Models Inference](https://mlsanity.com/paper/2407.16893/) (2024)
- [Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generation](https://mlsanity.com/paper/2310.08395/) (2023)
- [Benchmarking Vision-Language Models on Chinese Ancient Documents: From OCR to Knowledge Reasoning](https://mlsanity.com/paper/2509.09731/) (2025)
- [Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges](https://mlsanity.com/paper/2405.15604/) (2024)
- [Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models](https://mlsanity.com/paper/2401.08350/) (2024)

## Abstract (arXiv)

Large language models (LLMs) such as ChatGPT can produce coherent, cohesive, relevant, and fluent answers for various natural language processing (NLP) tasks. Taking document-level machine translation (MT) as a testbed, this paper provides an in-depth evaluation of LLMs' ability on discourse modeling. The study focuses on three aspects: 1) Effects of Context-Aware Prompts, where we investigate the impact of different prompts on document-level translation quality and discourse phenomena; 2) Comparison of Translation Models, where we compare the translation performance of ChatGPT with commercial MT systems and advanced document-level MT methods; 3) Analysis of Discourse Modelling Abilities, where we further probe discourse knowledge encoded in LLMs and shed light on impacts of training techniques on discourse modeling. By evaluating on a number of benchmarks, we surprisingly find that LLMs have demonstrated superior performance and show potential to become a new paradigm for document-level translation: 1) leveraging their powerful long-text modeling capabilities, GPT-3.5 and GPT-4 outperform commercial MT systems in terms of human evaluation; 2) GPT-4 demonstrates a stronger ability for probing linguistic knowledge than GPT-3.5. This work highlights the challenges and opportunities of LLMs for MT, which we hope can inspire the future design and evaluation of LLMs.We release our data and annotations at https://github.com/longyuewangdcu/Document-MT-LLM.