# Discourse Cohesion Evaluation for Document-Level Neural Machine Translation

arXiv: https://arxiv.org/abs/2208.09118 · HTML: https://mlsanity.com/paper/2208.09118/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 29 references that are in the map.
- First posted on arXiv 19 Aug 2022.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation](https://mlsanity.com/paper/1808.07048/) (2018)
- [Evaluating Discourse Phenomena in Neural Machine Translation](https://mlsanity.com/paper/1711.00513/) (2017)
- [Improving the Transformer Translation Model with Document-Level Context](https://mlsanity.com/paper/1810.03581/) (2018)
- [Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation](https://mlsanity.com/paper/1808.10432/) (2018)
- [Evaluating Pronominal Anaphora in Machine Translation: An Evaluation Measure and a Test Suite](https://mlsanity.com/paper/1909.00131/) (2019)
- [A Test Suite and Manual Evaluation of Document-Level NMT at WMT19](https://mlsanity.com/paper/1908.03043/) (2019)
- [SAO WMT19 Test Suite: Machine Translation of Audit Reports](https://mlsanity.com/paper/1909.01701/) (2019)

## Built on by (most cited first, 1 of 1)

- [How Far Can Machine Translation Quality Take You? Extrinsic Discourse Evaluation in Goal-Oriented Setups](https://mlsanity.com/paper/2606.16596/) (2026)

## Abstract (arXiv)

It is well known that translations generated by an excellent document-level neural machine translation (NMT) model are consistent and coherent. However, existing sentence-level evaluation metrics like BLEU can hardly reflect the model's performance at the document level. To tackle this issue, we propose a Discourse Cohesion Evaluation Method (DCoEM) in this paper and contribute a new test suite that considers four cohesive manners (reference, conjunction, substitution, and lexical cohesion) to measure the cohesiveness of document translations. The evaluation results on recent document-level NMT systems show that our method is practical and essential in estimating translations at the document level.