mlsanityData to 7 Oct 2026

OpenKiwi: An Open Source Framework for Quality Estimation

arXiv 1902.08646 ↗First posted 22 Feb 2019cs.CL cs.LGcited by 49

Not on the map. Why?

Abstract (arXiv)

We introduce OpenKiwi, a PyTorch-based open source framework for translation quality estimation. OpenKiwi supports training and testing of word-level and sentence-level quality estimation systems, implementing the winning systems of the WMT 2015-18 quality estimation campaigns. We benchmark OpenKiwi on two datasets from WMT 2018 (English-German SMT and NMT), yielding state-of-the-art performance on the word-level tasks and near state-of-the-art in the sentence-level tasks.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 13; 2022: 10; 2023: 9; 2024: 10; 2025: 4; 2026: 3.

49 papers.

Sort:

  1. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 205
  2. 2022Quality-Aware Decoding for Neural Machine Translationcited by 49
  3. 2021Uncertainty-Aware Machine Translation Evaluationcited by 35
  4. 2022UniTE: Unified Translation Evaluationcited by 28
  5. 2021BLEU, METEOR, BERTScore: Evaluation of Metrics Performance in Assessing Critical Translation Errors in Sentiment-oriented Textcited by 22
  6. 2023Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errorscited by 22
  7. 2023There's no Data Like Better Data: Using QE Metrics for MT Data Filteringcited by 19
  8. 2021An Exploratory Analysis of Multilingual Word-Level Quality Estimation with Cross-Lingual Transformerscited by 18
  9. 2024What Makes a Good Story and How Can We Measure It? A Comprehensive Survey of Story Evaluationcited by 17
  10. 2021Quality Estimation without Human-labeled Datacited by 15
  11. 2024MT-Ranker: Reference-free machine translation evaluation by inter-system rankingcited by 14
  12. 2022As Little as Possible, as Much as Necessary: Detecting Over- and Undertranslations with Contrastive Conditioningcited by 11
37 more
  1. 2023Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Referencecited by 11
  2. 2024From Handcrafted Features to LLMs: A Brief Survey for Machine Translation Quality Estimationcited by 10
  3. 2024SpeechQE: Estimating the Quality of Direct Speech Translationcited by 10
  4. 2025Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progresscited by 10
  5. 2022Disentangling Uncertainty in Machine Translation Evaluationcited by 9
  6. 2022SESCORE2: Learning Text Generation Evaluation via Synthesizing Realistic Mistakescited by 9
  7. 2023Quality-Aware Translation Models: Efficient Generation and Quality Estimation in a Single Modelcited by 9
  8. 2022Better Quality Estimation for Low Resource Corpus Miningcited by 8
  9. 2025QE4PE: Word-level Quality Estimation for Human Post-Editingcited by 6
  10. 2021Knowledge Distillation for Quality Estimationcited by 5
  11. 2021Pushing the Right Buttons: Adversarial Evaluation of Quality Estimationcited by 4
  12. 2023Towards Fine-Grained Information: Identifying the Type and Location of Translation Errorscited by 4
  13. 2024Multi-Dimensional Machine Translation Evaluation: Model Evaluation and Resource for Koreancited by 4
  14. 2024How to Learn in a Noisy World? Self-Correcting the Real-World Data Noise in Machine Translationcited by 4
  15. 2021Levenshtein Training for Word-level Quality Estimationcited by 3
  16. 2021EXPATS: A Toolkit for Explainable Automated Text Scoringcited by 2
  17. 2021Ensemble Fine-tuned mBERT for Translation Quality Estimationcited by 2
  18. 2022Rethink about the Word-level Quality Estimation for Machine Translation from Human Judgementcited by 2
  19. 2022Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality?cited by 2
  20. 2023KG-BERTScore: Incorporating Knowledge Graph into BERTScore for Reference-Free Machine Translation Evaluationcited by 2
  21. 2024Multi-Target Cross-Lingual Summarization: a novel task and a language-neutral approachcited by 2
  22. 2025Penalizing Length: Uncovering Systematic Bias in Quality Estimation Metricscited by 2
  23. 2021Detecting over/under-translation errors for determining adequacy in human translationscited by 1
  24. 2021Verdi: Quality Estimation and Error Detection for Bilingual Corporacited by 1
  25. 2021The JHU-Microsoft Submission for WMT21 Quality Estimation Shared Taskcited by 1
  26. 2022Mismatching-Aware Unsupervised Translation Quality Estimation For Low-Resource Languagescited by 1
  27. 2023Negative Lexical Constraints in Neural Machine Translationcited by 1
  28. 2023EMMA-X: An EM-like Multilingual Pre-training Algorithm for Cross-lingual Representation Learningcited by 1
  29. 2023SurreyAI 2023 Submission for the Quality Estimation Shared Taskcited by 1
  30. 2024BiVert: Bidirectional Vocabulary Evaluation using Relations for Machine Translationcited by 1
  31. 2024A Multi-task Learning Framework for Evaluating Machine Translation of Emotion-loaded User-generated Contentcited by 1
  32. 2026Semi-Synthetic Parallel Data for Translation Quality Estimation: A Case Study of Dataset Building for an Under-Resourced Language Paircited by 1
  33. 2026Can We Triage LLM Translation Errors in Classical Texts Without Human References? Source Novelty, GEMBA Scoring, and Budgeted Review through Pali-to-English Translationcited by 1
  34. 2021Ensemble-based Transfer Learning for Low-resource Machine Translation Quality Estimationcited by 0
  35. 2024Quantity vs. Quality of Monolingual Source Data in Automatic Text Translation: Can It Be Too Little If It Is Too Good?cited by 0
  36. 2025ALOPE: Adaptive Layer Optimization for Translation Quality Estimation using Large Language Modelscited by 0
  37. 2026Smarter edits? Post-editing with error highlights and translation suggestionscited by 0