# PreQuEL: Quality Estimation of Machine Translation Outputs in Advance

arXiv: https://arxiv.org/abs/2205.09178 · HTML: https://mlsanity.com/paper/2205.09178/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 18 May 2022.

## Builds on (most cited first, 24 of 24)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping](https://mlsanity.com/paper/2002.06305/) (2020)
- [Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection](https://mlsanity.com/paper/2004.10643/) (2020)
- [Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks](https://mlsanity.com/paper/1811.01088/) (2018)
- [Marian: Fast Neural Machine Translation in C++](https://mlsanity.com/paper/1804.00344/) (2018)
- [Muppet: Massive Multi-task Representations with Pre-Finetuning](https://mlsanity.com/paper/2101.11038/) (2021)
- [Facebook FAIR's WMT19 News Translation Task Submission](https://mlsanity.com/paper/1907.06616/) (2019)
- [To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation](https://mlsanity.com/paper/2107.10821/) (2021)
- [The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT](https://mlsanity.com/paper/2010.06354/) (2020)
- [NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation](https://mlsanity.com/paper/2112.02721/) (2021)
- [MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset](https://mlsanity.com/paper/2010.04480/) (2020)
- [A Challenge Set Approach to Evaluating Machine Translation](https://mlsanity.com/paper/1704.07431/) (2017)
- [TransQuest at WMT2020: Sentence-Level Direct Assessment](https://mlsanity.com/paper/2010.05318/) (2020)
- [Inherent Biases in Reference based Evaluation for Grammatical Error Correction and Text Simplification](https://mlsanity.com/paper/1804.11254/) (2018)
- [Automatically Extracting Challenge Sets for Non local Phenomena in Neural Machine Translation](https://mlsanity.com/paper/1909.06814/) (2019)
- [A Multifaceted Evaluation of Neural versus Phrase-Based Machine Translation for 9 Language Directions](https://mlsanity.com/paper/1701.02901/) (2017)
- [Unsupervised Identification of Translationese](https://mlsanity.com/paper/1609.03205/) (2016)
- [Fine-Grained Analysis of Cross-Linguistic Syntactic Divergences](https://mlsanity.com/paper/2005.03436/) (2020)
- [Enhancing the Transformer Decoder with Transition-based Syntax](https://mlsanity.com/paper/2101.12640/) (2021)

## Built on by (most cited first, 9 of 9)

- [Where to start? Analyzing the potential value of intermediate models](https://mlsanity.com/paper/2211.00107/) (2022)
- [How to Select Datapoints for Efficient Human Evaluation of NLG Models?](https://mlsanity.com/paper/2501.18251/) (2025)
- [Label-Efficient Model Selection for Text Generation](https://mlsanity.com/paper/2402.07891/) (2024)
- [Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference](https://mlsanity.com/paper/2301.09008/) (2023)
- [Human Learning by Model Feedback: The Dynamics of Iterative Prompting with Midjourney](https://mlsanity.com/paper/2311.12131/) (2023)
- [Textual Similarity as a Key Metric in Machine Translation Quality Estimation](https://mlsanity.com/paper/2406.07440/) (2024)
- [RouteLMT: Learned Sample Routing for Hybrid LLM Translation Deployment](https://mlsanity.com/paper/2604.22520/) (2026)
- [Quality Estimation of Machine Translated Texts based on Direct Evidence from Training Data](https://mlsanity.com/paper/2306.15399/) (2023)
- [PreSumm: Predicting Summarization Performance Without Summarizing](https://mlsanity.com/paper/2504.05420/) (2025)

## Abstract (arXiv)

We present the task of PreQuEL, Pre-(Quality-Estimation) Learning. A PreQuEL system predicts how well a given sentence will be translated, without recourse to the actual translation, thus eschewing unnecessary resource allocation when translation quality is bound to be low. PreQuEL can be defined relative to a given MT system (e.g., some industry service) or generally relative to the state-of-the-art. From a theoretical perspective, PreQuEL places the focus on the source text, tracing properties, possibly linguistic features, that make a sentence harder to machine translate. We develop a baseline model for the task and analyze its performance. We also develop a data augmentation method (from parallel corpora), that improves results substantially. We show that this augmentation method can improve the performance of the Quality-Estimation task as well. We investigate the properties of the input text that our model is sensitive to, by testing it on challenge sets and different languages. We conclude that it is aware of syntactic and semantic distinctions, and correlates and even over-emphasizes the importance of standard NLP features.