mlsanityData to 14 Oct 2026

Multilingual Contextualization of Large Language Models for Document-Level Machine Translation

arXiv 2504.12140 ↗First posted 16 Apr 2025cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have demonstrated strong performance in sentence-level machine translation, but scaling to document-level translation remains challenging, particularly in modeling long-range dependencies and discourse phenomena across sentences and paragraphs. In this work, we propose a method to improve LLM-based long-document translation through targeted fine-tuning on high-quality document-level data, which we curate and introduce as DocBlocks. Our approach supports multiple translation paradigms, including direct document-to-document and chunk-level translation, by integrating instructions both with and without surrounding context. This enables models to better capture cross-sentence dependencies while maintaining strong sentence-level translation performance. Experimental results show that incorporating multiple translation paradigms improves document-level translation quality and inference speed compared to prompting and agent-based methods.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 50 of its 78 references that are in the map. First posted on arXiv 16 Apr 2025.

Builds on

50 of 78 references.

References · 50

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2023Qwen Technical Reportcited by 3,344
  9. 2023Mistral 7Bcited by 3,164
  10. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  11. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  12. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
38 more
  1. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  2. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  3. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  4. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  5. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 250
  6. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  7. 2024EuroLLM: Multilingual Language Models for Europecited by 106
  8. 2018Context-Aware Neural Machine Translation Learns Anaphora Resolutioncited by 97
  9. 2023Document-Level Machine Translation with Large Language Modelscited by 90
  10. 2023A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Modelscited by 89
  11. 2018Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluationcited by 88
  12. 2023Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Taskcited by 88
  13. 2017Evaluating Discourse Phenomena in Neural Machine Translationcited by 87
  14. 2018Document-Level Neural Machine Translation with Hierarchical Attention Networkscited by 85
  15. 2023Large language models effectively leverage document-level context for literary translation, but critical errors persistcited by 80
  16. 2025WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialectscited by 79
  17. 2018Improving the Transformer Translation Model with Document-Level Contextcited by 74
  18. 2020TICO-19: the Translation Initiative for Covid-19cited by 68
  19. 2021Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translationcited by 65
  20. 2024Adapting Large Language Models for Document-Level Machine Translationcited by 59
  21. 2021BlonDe: An Automatic Evaluation Metric for Document-level Machine Translationcited by 58
  22. 2022Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric Into a Document-Level Metriccited by 56
  23. 2024(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Textscited by 49
  24. 2021Measuring and Increasing Context Usage in Context-Aware Machine Translationcited by 48
  25. 2023Exploring Human-Like Translation Strategy with Large Language Modelscited by 44
  26. 2020Rethinking Document-level Neural Machine Translationcited by 32
  27. 2020Long-Short Term Masking Transformer: A Simple but Effective Baseline for Document-level Neural Machine Translationcited by 27
  28. 2024DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memorycited by 27
  29. 2020Diving Deep into Context-Aware Neural Machine Translationcited by 25
  30. 2023Findings of the WMT 2023 Shared Task on Discourse-Level Literary Translation: A Fresh Orb in the Cosmos of LLMscited by 25
  31. 2020Reassessing Claims of Human Parity and Super-Human Performance in Machine Translation at WMT 2019cited by 20
  32. 2018Modeling Coherence for Discourse Neural Machine Translationcited by 15
  33. 2023Document Flattening: Beyond Concatenating Context for Document-Level Neural Machine Translationcited by 15
  34. 2024Enhancing Document-level Translation of Large Language Model via Translation Mixed-instructionscited by 13
  35. 2018What Level of Quality can Neural Machine Translation Attain on Literary Text?cited by 11
  36. 2023SLIDE: Reference-free Evaluation for Machine Translation using a Sliding Document Windowcited by 11
  37. 2019Using Whole Document Context in Neural Machine Translationcited by 9
  38. 2024Importance-Aware Data Augmentation for Document-Level Neural Machine Translationcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 6.

11 papers.

  1. 2025DocHPLT: A Massively Multilingual Document-Level Translation Datasetcited by 7
  2. 2025HPLT 3.0: Very Large-Scale Multilingual Resources for LLMs and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Modelscited by 7
  3. 2026Unveiling Language Routing Isolation in Multilingual MoE Models for Interpretable Subnetwork Adaptationcited by 4
  4. 2026Cross-Preference Learning for Sentence-Level and Context-Aware Machine Translationcited by 2
  5. 2026What Does LLM Refinement Actually Improve? A Systematic Study on Document-Level Literary Translationcited by 2
  6. 2025ACADATA: Parallel Dataset of Academic Data for Machine Translationcited by 1
  7. 2025Discourse Graph Guided Document Translation with Large Language Modelscited by 1
  8. 2026Loong: A Human-Like Long Document Translation Agent with Observe-and-Act Adaptive Context Selectioncited by 1
  9. 2025Unlocking Latent Discourse Translation in LLMs Through Quality-Aware Decodingcited by 0
  10. 2026DataRx: Missingness-Aware Sampling for Safer Large Language Model Task-Specific Fine-Tuningcited by 0
  11. 2026Doc2FRC: Length-Consistent Document-Level Machine Translation via Fixed-Range Chunkingcited by 0