# Investigating Length Issues in Document-level Machine Translation

arXiv: https://arxiv.org/abs/2412.17592 · HTML: https://mlsanity.com/paper/2412.17592/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 85 references that are in the map.
- First posted on arXiv 23 Dec 2024.

## Builds on (most cited first, 30 of 55)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation](https://mlsanity.com/paper/2108.12409/) (2021)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [Landmark Attention: Random-Access Infinite Context Length for Transformers](https://mlsanity.com/paper/2305.16300/) (2023)
- [Tower: An Open Multilingual Large Language Model for Translation-Related Tasks](https://mlsanity.com/paper/2402.17733/) (2024)
- [Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics](https://mlsanity.com/paper/2006.06264/) (2020)
- [Randomized Positional Encodings Boost Length Generalization of Transformers](https://mlsanity.com/paper/2305.16843/) (2023)
- [Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing](https://mlsanity.com/paper/2004.14564/) (2020)
- [PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training](https://mlsanity.com/paper/2309.10400/) (2023)
- [Context-Aware Neural Machine Translation Learns Anaphora Resolution](https://mlsanity.com/paper/1805.10163/) (2018)
- [Neural Machine Translation with Extended Context](https://mlsanity.com/paper/1708.05943/) (2017)
- [Evaluating Discourse Phenomena in Neural Machine Translation](https://mlsanity.com/paper/1711.00513/) (2017)
- [Document-Level Neural Machine Translation with Hierarchical Attention Networks](https://mlsanity.com/paper/1809.01576/) (2018)
- [Large language models effectively leverage document-level context for literary translation, but critical errors persist](https://mlsanity.com/paper/2304.03245/) (2023)
- [Improving the Transformer Translation Model with Document-Level Context](https://mlsanity.com/paper/1810.03581/) (2018)
- [Selective Attention for Context-aware Neural Machine Translation](https://mlsanity.com/paper/1903.08788/) (2019)

## Built on by (most cited first, 10 of 10)

- [DocHPLT: A Massively Multilingual Document-Level Translation Dataset](https://mlsanity.com/paper/2508.13079/) (2025)
- [Optimising ChatGPT for creativity in literary translation: A case study from English into Dutch, Chinese, Catalan and Spanish](https://mlsanity.com/paper/2504.18221/) (2025)
- [XToM: Exploring the Multilingual Theory of Mind for Large Language Models](https://mlsanity.com/paper/2506.02461/) (2025)
- [What Does LLM Refinement Actually Improve? A Systematic Study on Document-Level Literary Translation](https://mlsanity.com/paper/2605.13368/) (2026)
- [STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translation](https://mlsanity.com/paper/2608.27161/) (2026)
- [Hindsight Quality Prediction Experiments in Multi-Candidate Human-Post-Edited Machine Translation](https://mlsanity.com/paper/2603.04083/) (2026)
- [Failure-Aware Long-Form Translation: Design and Implementation of a Recoverable LLM Translation System](https://mlsanity.com/paper/2608.09187/) (2026)
- [Doc2FRC: Length-Consistent Document-Level Machine Translation via Fixed-Range Chunking](https://mlsanity.com/paper/2609.12674/) (2026)
- [When Do Agents Help? Embedding, LLM and Agentic Alignment of Classical Texts and Their Translations](https://mlsanity.com/paper/2609.33691/) (2026)
- [TermJudge: A Document-Level Metric Judging, Not Counting, Terminology in Machine Translation Evaluation](https://mlsanity.com/paper/2609.35017/) (2026)

## Abstract (arXiv)

Transformer architectures are increasingly effective at processing and generating very long chunks of texts, opening new perspectives for document-level machine translation (MT). In this work, we challenge the ability of MT systems to handle texts comprising up to several thousands of tokens. We design and implement a new approach designed to precisely measure the effect of length increments on MT outputs. Our experiments with two representative architectures unambiguously show that (a)~translation performance decreases with the length of the input text; (b)~the position of sentences within the document matters, and translation quality is higher for sentences occurring earlier in a document. We further show that manipulating the distribution of document lengths and of positional embeddings only marginally mitigates such problems. Our results suggest that even though document-level MT is computationally feasible, it does not yet match the performance of sentence-based MT.