# SLIDE: Reference-free Evaluation for Machine Translation using a Sliding Document Window

arXiv: https://arxiv.org/abs/2309.08832 · HTML: https://mlsanity.com/paper/2309.08832/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 23 references that are in the map.
- First posted on arXiv 16 Sep 2023.

## Builds on (most cited first, 15 of 15)

- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training](https://mlsanity.com/paper/2007.07834/) (2020)
- [CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task](https://mlsanity.com/paper/2209.06243/) (2022)
- [To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation](https://mlsanity.com/paper/2107.10821/) (2021)
- [Evaluating Discourse Phenomena in Neural Machine Translation](https://mlsanity.com/paper/1711.00513/) (2017)
- [Larger-Scale Transformers for Multilingual Masked Language Modeling](https://mlsanity.com/paper/2105.00572/) (2021)
- [BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation](https://mlsanity.com/paper/2103.11878/) (2021)
- [Unbabel's Participation in the WMT20 Metrics Shared Task](https://mlsanity.com/paper/2010.15535/) (2020)
- [Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric Into a Document-Level Metric](https://mlsanity.com/paper/2209.13654/) (2022)
- [When Does Translation Require Context? A Data-driven, Multilingual Exploration](https://mlsanity.com/paper/2109.07446/) (2021)
- [Escaping the sentence-level paradigm in machine translation](https://mlsanity.com/paper/2304.12959/) (2023)
- [A Survey on Document-level Neural Machine Translation: Methods and Evaluation](https://mlsanity.com/paper/1912.08494/) (2019)
- [Training and Meta-Evaluating Machine Translation Evaluation Metrics at the Paragraph Level](https://mlsanity.com/paper/2308.13506/) (2023)

## Built on by (most cited first, 11 of 11)

- [Improving Statistical Significance in Human Evaluation of Automatic Metrics via Soft Pairwise Accuracy](https://mlsanity.com/paper/2409.09598/) (2024)
- [Multilingual Contextualization of Large Language Models for Document-Level Machine Translation](https://mlsanity.com/paper/2504.12140/) (2025)
- [Adapters for Altering LLM Vocabularies: What Languages Benefit the Most?](https://mlsanity.com/paper/2410.09644/) (2024)
- [Recovering document annotations for sentence-level bitext](https://mlsanity.com/paper/2406.03869/) (2024)
- [Extending Automatic Machine Translation Evaluation to Book-Length Documents](https://mlsanity.com/paper/2509.17249/) (2025)
- [Align-then-Slide: A complete evaluation framework for Ultra-Long Document-Level Machine Translation](https://mlsanity.com/paper/2509.03809/) (2025)
- [Long-context Reference-based MT Quality Estimation](https://mlsanity.com/paper/2509.13980/) (2025)
- [Same evaluation, more tokens: On the effect of input length for machine translation evaluation using Large Language Models](https://mlsanity.com/paper/2505.01761/) (2025)
- [Quality Estimation Reranking for Document-Level Translation](https://mlsanity.com/paper/2510.08870/) (2025)
- [ACADATA: Parallel Dataset of Academic Data for Machine Translation](https://mlsanity.com/paper/2510.12621/) (2025)
- [Doc2FRC: Length-Consistent Document-Level Machine Translation via Fixed-Range Chunking](https://mlsanity.com/paper/2609.12674/) (2026)

## Abstract (arXiv)

Reference-based metrics that operate at the sentence-level typically outperform quality estimation metrics, which have access only to the source and system output. This is unsurprising, since references resolve ambiguities that may be present in the source. In this paper, we investigate whether additional source context can effectively substitute for a reference. We present a metric named SLIDE (SLIding Document Evaluator), which operates on blocks of sentences. SLIDE leverages a moving window that slides over each document in the test set, feeding each chunk of sentences into an unmodified, off-the-shelf quality estimation model. We find that SLIDE obtains significantly higher pairwise system accuracy than its sentence-level baseline, in some cases even eliminating the gap with reference-base metrics. This suggests that source context may provide the same information as a human reference in disambiguating source ambiguities. This finding is especially pertinent for reference-free document-level evaluation, wherein SLIDE could provide higher-quality pairwise system assessments while only requiring document boundary annotations.