# Beyond Glass-Box Features: Uncertainty Quantification Enhanced Quality Estimation for Neural Machine Translation

arXiv: https://arxiv.org/abs/2109.07141 · HTML: https://mlsanity.com/paper/2109.07141/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 27 references that are in the map.
- First posted on arXiv 15 Sep 2021.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [How multilingual is Multilingual BERT?](https://mlsanity.com/paper/1906.01502/) (2019)
- [Unsupervised Quality Estimation for Neural Machine Translation](https://mlsanity.com/paper/2005.10608/) (2020)
- [Nearest Neighbor Machine Translation](https://mlsanity.com/paper/2010.00710/) (2020)
- [The Effect of Translationese in Machine Translation Test Sets](https://mlsanity.com/paper/1906.08069/) (2019)
- [MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset](https://mlsanity.com/paper/2010.04480/) (2020)
- [TransQuest at WMT2020: Sentence-Level Direct Assessment](https://mlsanity.com/paper/2010.05318/) (2020)
- [Quality Estimation without Human-labeled Data](https://mlsanity.com/paper/2102.04020/) (2021)
- [Unbabel's Participation in the WMT19 Translation Quality Estimation Shared Task](https://mlsanity.com/paper/1907.10352/) (2019)
- ["Bilingual Expert" Can Find Translation Errors](https://mlsanity.com/paper/1807.09433/) (2018)
- [Computer Assisted Translation with Neural Quality Estimation and Automatic Post-Editing](https://mlsanity.com/paper/2009.09126/) (2020)

## Built on by (most cited first, 5 of 5)

- [AfriMTE and AfriCOMET: Enhancing COMET to Embrace Under-resourced African Languages](https://mlsanity.com/paper/2311.09828/) (2023)
- [BLEU Meets COMET: Combining Lexical and Neural Metrics Towards Robust Machine Translation Evaluation](https://mlsanity.com/paper/2305.19144/) (2023)
- [QEMind: Alibaba's Submission to the WMT21 Quality Estimation Shared Task](https://mlsanity.com/paper/2112.14890/) (2021)
- [Quality Estimation with k-nearest Neighbors and Automatic Evaluation for Model-specific Quality Estimation](https://mlsanity.com/paper/2404.18031/) (2024)
- [Learning Confidence for Transformer-based Neural Machine Translation](https://mlsanity.com/paper/2203.11413/) (2022)

## Abstract (arXiv)

Quality Estimation (QE) plays an essential role in applications of Machine Translation (MT). Traditionally, a QE system accepts the original source text and translation from a black-box MT system as input. Recently, a few studies indicate that as a by-product of translation, QE benefits from the model and training data's information of the MT system where the translations come from, and it is called the "glass-box QE". In this paper, we extend the definition of "glass-box QE" generally to uncertainty quantification with both "black-box" and "glass-box" approaches and design several features deduced from them to blaze a new trial in improving QE's performance. We propose a framework to fuse the feature engineering of uncertainty quantification into a pre-trained cross-lingual language model to predict the translation quality. Experiment results show that our method achieves state-of-the-art performances on the datasets of WMT 2020 QE shared task.