# "Bilingual Expert" Can Find Translation Errors

arXiv: https://arxiv.org/abs/1807.09433 · HTML: https://mlsanity.com/paper/1807.09433/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2018.

## Built on by (most cited first, 10 of 10)

- [AfriMTE and AfriCOMET: Enhancing COMET to Embrace Under-resourced African Languages](https://mlsanity.com/paper/2311.09828/) (2023)
- [From Handcrafted Features to LLMs: A Brief Survey for Machine Translation Quality Estimation](https://mlsanity.com/paper/2403.14118/) (2024)
- [DirectQE: Direct Pretraining for Machine Translation Quality Estimation](https://mlsanity.com/paper/2105.07149/) (2021)
- [Beyond Glass-Box Features: Uncertainty Quantification Enhanced Quality Estimation for Neural Machine Translation](https://mlsanity.com/paper/2109.07141/) (2021)
- [QEMind: Alibaba's Submission to the WMT21 Quality Estimation Shared Task](https://mlsanity.com/paper/2112.14890/) (2021)
- [Rethink about the Word-level Quality Estimation for Machine Translation from Human Judgement](https://mlsanity.com/paper/2209.05695/) (2022)
- [Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality?](https://mlsanity.com/paper/2211.13865/) (2022)
- [Verdi: Quality Estimation and Error Detection for Bilingual Corpora](https://mlsanity.com/paper/2105.14878/) (2021)
- [Ensemble-based Transfer Learning for Low-resource Machine Translation Quality Estimation](https://mlsanity.com/paper/2105.07622/) (2021)
- [MDQE: A More Accurate Direct Pretraining for Machine Translation Quality Estimation](https://mlsanity.com/paper/2107.14600/) (2021)

## Abstract (arXiv)

Recent advances in statistical machine translation via the adoption of neural sequence-to-sequence models empower the end-to-end system to achieve state-of-the-art in many WMT benchmarks. The performance of such machine translation (MT) system is usually evaluated by automatic metric BLEU when the golden references are provided for validation. However, for model inference or production deployment, the golden references are prohibitively available or require expensive human annotation with bilingual expertise. In order to address the issue of quality evaluation (QE) without reference, we propose a general framework for automatic evaluation of translation output for most WMT quality evaluation tasks. We first build a conditional target language model with a novel bidirectional transformer, named neural bilingual expert model, which is pre-trained on large parallel corpora for feature extraction. For QE inference, the bilingual expert model can simultaneously produce the joint latent representation between the source and the translation, and real-valued measurements of possible erroneous tokens based on the prior knowledge learned from parallel data. Subsequently, the features will further be fed into a simple Bi-LSTM predictive model for quality evaluation. The experimental results show that our approach achieves the state-of-the-art performance in the quality estimation track of WMT 2017/2018.