mlsanityData to 14 Oct 2026

Quality Estimation without Human-labeled Data

arXiv 2102.04020 ↗First posted 8 Feb 2021cs.CLcited by 15

Not on the map. Why?

Abstract (arXiv)

Quality estimation aims to measure the quality of translated content without access to a reference translation. This is crucial for machine translation systems in real-world scenarios where high-quality translation is needed. While many approaches exist for quality estimation, they are based on supervised machine learning requiring costly human labelled data. As an alternative, we propose a technique that does not rely on examples from human-annotators and instead uses synthetic training data. We train off-the-shelf architectures for supervised quality estimation on our synthetic data and show that the resulting models achieve comparable performance to models trained on human-annotated data, both for sentence and word-level prediction.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 14 of its 29 references that are in the map. First posted on arXiv 8 Feb 2021.

Builds on

14 of 29 references.

References · 14

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  4. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  5. 2018Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyondcited by 434
  6. 2019WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipediacited by 223
  7. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 211
  8. 2019CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEBcited by 177
  9. 2019CCAligned: A Massive Collection of Cross-Lingual Web-Document Pairscited by 136
  10. 2019OpenKiwi: An Open Source Framework for Quality Estimationcited by 49
  11. 2019Text Infillingcited by 32
  12. 2018eSCAPE: a Large-scale Synthetic Corpus for Automatic Post-Editingcited by 18
2 more
  1. 2019Unbabel's Participation in the WMT19 Translation Quality Estimation Shared Taskcited by 13
  2. 2020Massively Multilingual Document Alignment with Cross-lingual Sentence-Mover's Distancecited by 7

Built on by

Cited by year

Citing papers per year: 2021: 4; 2022: 6; 2023: 2; 2024: 0; 2025: 3.

15 papers.

Sort:

By area:

  1. 2021The Eval4NLP Shared Task on Explainable Quality Estimation: Overview and Resultscited by 27
  2. 2021Translation Error Detection as Rationale Extractioncited by 22
  3. 2022Not All Errors are Equal: Learning Text Generation Metrics using Stratified Error Synthesiscited by 18
  4. 2022As Little as Possible, as Much as Necessary: Detecting Over- and Undertranslations with Contrastive Conditioningcited by 11
  5. 2023Perturbation-based QE: An Explainable, Unsupervised Word-level Quality Estimation Method for Blackbox Machine Translationcited by 6
  6. 2021Beyond Glass-Box Features: Uncertainty Quantification Enhanced Quality Estimation for Neural Machine Translationcited by 5
  7. 2025Are Generative Models Underconfident? Better Quality Estimation with Boosted Model Probabilitycited by 4
  8. 2025Alleviating Distribution Shift in Synthetic Data for Machine Translation Quality Estimationcited by 4
  9. 2022Rethink about the Word-level Quality Estimation for Machine Translation from Human Judgementcited by 2
  10. 2022Advancing Semi-Supervised Learning for Automatic Post-Editing: Data-Synthesis by Mask-Infilling with Erroneous Termscited by 1
  11. 2022Mismatching-Aware Unsupervised Translation Quality Estimation For Low-Resource Languagescited by 1
  12. 2021Classification-based Quality Estimation: Small and Efficient Models for Real-world Applicationscited by 0
3 more
  1. 2022QUAK: A Synthetic Quality Estimation Dataset for Korean-English Neural Machine Translationcited by 0
  2. 2023Unsupervised Translation Quality Estimation Exploiting Synthetic Data and Pre-trained Multilingual Encodercited by 0
  3. 2025Evaluating Language Translation Models by Playing Telephonecited by 0