# DirectQE: Direct Pretraining for Machine Translation Quality Estimation

arXiv: https://arxiv.org/abs/2105.07149 · HTML: https://mlsanity.com/paper/2105.07149/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 15 May 2021.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives](https://mlsanity.com/paper/1909.01380/) (2019)
- [Language GANs Falling Short](https://mlsanity.com/paper/1811.02549/) (2018)
- [eSCAPE: a Large-scale Synthetic Corpus for Automatic Post-Editing](https://mlsanity.com/paper/1803.07274/) (2018)
- [Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing](https://mlsanity.com/paper/1605.04800/) (2016)
- [Unbabel's Participation in the WMT19 Translation Quality Estimation Shared Task](https://mlsanity.com/paper/1907.10352/) (2019)
- ["Bilingual Expert" Can Find Translation Errors](https://mlsanity.com/paper/1807.09433/) (2018)

## Built on by (most cited first, 7 of 7)

- [Cross-Lingual Cross-Modal Retrieval with Noise-Robust Learning](https://mlsanity.com/paper/2208.12526/) (2022)
- [Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference](https://mlsanity.com/paper/2301.09008/) (2023)
- [From Handcrafted Features to LLMs: A Brief Survey for Machine Translation Quality Estimation](https://mlsanity.com/paper/2403.14118/) (2024)
- [Alleviating Distribution Shift in Synthetic Data for Machine Translation Quality Estimation](https://mlsanity.com/paper/2502.19941/) (2025)
- [Rethink about the Word-level Quality Estimation for Machine Translation from Human Judgement](https://mlsanity.com/paper/2209.05695/) (2022)
- [Unify word-level and span-level tasks: NJUNLP's Participation for the WMT2023 Quality Estimation Shared Task](https://mlsanity.com/paper/2309.13230/) (2023)
- [MDQE: A More Accurate Direct Pretraining for Machine Translation Quality Estimation](https://mlsanity.com/paper/2107.14600/) (2021)

## Abstract (arXiv)

Machine Translation Quality Estimation (QE) is a task of predicting the quality of machine translations without relying on any reference. Recently, the predictor-estimator framework trains the predictor as a feature extractor, which leverages the extra parallel corpora without QE labels, achieving promising QE performance. However, we argue that there are gaps between the predictor and the estimator in both data quality and training objectives, which preclude QE models from benefiting from a large number of parallel corpora more directly. We propose a novel framework called DirectQE that provides a direct pretraining for QE tasks. In DirectQE, a generator is trained to produce pseudo data that is closer to the real QE data, and a detector is pretrained on these data with novel objectives that are akin to the QE task. Experiments on widely used benchmarks show that DirectQE outperforms existing methods, without using any pretraining models such as BERT. We also give extensive analyses showing how fixing the two gaps contributes to our improvements.