# Translation Quality Assessment: A Brief Survey on Manual and Automatic Methods

arXiv: https://arxiv.org/abs/2105.03311 · HTML: https://mlsanity.com/paper/2105.03311/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 147 references that are in the map.
- First posted on arXiv 5 May 2021.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics](https://mlsanity.com/paper/2102.01672/) (2021)
- [Unsupervised Quality Estimation for Neural Machine Translation](https://mlsanity.com/paper/2005.10608/) (2020)
- [XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation](https://mlsanity.com/paper/2104.07412/) (2021)
- [Re-evaluating Evaluation in Text Summarization](https://mlsanity.com/paper/2010.07100/) (2020)
- [Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation](https://mlsanity.com/paper/1808.07048/) (2018)
- [A Set of Recommendations for Assessing Human-Machine Parity in Language Translation](https://mlsanity.com/paper/2004.01694/) (2020)
- [Pairwise Neural Machine Translation Evaluation](https://mlsanity.com/paper/1912.03135/) (2019)
- [Chinese Character Decomposition for Neural MT with Multi-Word Expressions](https://mlsanity.com/paper/2104.04497/) (2021)
- [LEPOR: An Augmented Machine Translation Evaluation Metric](https://mlsanity.com/paper/1703.08748/) (2017)
- [MultiMWE: Building a Multi-lingual Multi-Word Expression (MWE) Parallel Corpora](https://mlsanity.com/paper/2005.10583/) (2020)
- [Machine Translation Evaluation with Neural Networks](https://mlsanity.com/paper/1710.02095/) (2017)

## Built on by (most cited first, 23 of 23)

- [Physician Detection of Clinical Harm in Machine Translation: Quality Estimation Aids in Reliance and Backtranslation Identifies Critical Errors](https://mlsanity.com/paper/2310.16924/) (2023)
- [HOPE: A Task-Oriented and Human-Centric Evaluation Framework Using Professional Post-Editing Towards More Effective MT Evaluation](https://mlsanity.com/paper/2112.13833/) (2021)
- [Measuring Uncertainty in Translation Quality Evaluation (TQE)](https://mlsanity.com/paper/2111.07699/) (2021)
- [Who Evaluates the Evaluators? On Automatic Metrics for Assessing AI-based Offensive Code Generators](https://mlsanity.com/paper/2212.06008/) (2022)
- [Evaluating Task-oriented Dialogue Systems: A Systematic Review of Measures, Constructs and their Operationalisations](https://mlsanity.com/paper/2312.13871/) (2023)
- [cushLEPOR: customising hLEPOR metric using Optuna for higher agreement with human judgments or pre-trained language model LaBSE](https://mlsanity.com/paper/2108.09484/) (2021)
- [Investigating Large Language Models and Control Mechanisms to Improve Text Readability of Biomedical Abstracts](https://mlsanity.com/paper/2309.13202/) (2023)
- [Neural Machine Translation of Clinical Text: An Empirical Investigation into Multilingual Pre-Trained Language Models and Transfer-Learning](https://mlsanity.com/paper/2312.07250/) (2023)
- [AskQE: Question Answering as Automatic Evaluation for Machine Translation](https://mlsanity.com/paper/2504.11582/) (2025)
- [Automating the Correctness Assessment of AI-generated Code for Security Contexts](https://mlsanity.com/paper/2310.18834/) (2023)
- [Advancing Dialectal Arabic to Modern Standard Arabic Machine Translation](https://mlsanity.com/paper/2507.20301/) (2025)
- [Is Context Helpful for Chat Translation Evaluation?](https://mlsanity.com/paper/2403.08314/) (2024)
- [An Overview on Machine Translation Evaluation](https://mlsanity.com/paper/2202.11027/) (2022)
- [Enhancing Robustness of AI Offensive Code Generators via Data Augmentation](https://mlsanity.com/paper/2306.05079/) (2023)
- [Can NMT Understand Me? Towards Perturbation-based Evaluation of NMT Models for Code Generation](https://mlsanity.com/paper/2203.15319/) (2022)
- [Student's t-Distribution: On Measuring the Inter-Rater Reliability When the Observations are Scarce](https://mlsanity.com/paper/2303.04526/) (2023)
- [RELISH: LLM REgression with a Latent Iterative State Head](https://mlsanity.com/paper/2604.01206/) (2026)
- [A new approach to calculating BERTScore for automatic assessment of translation quality](https://mlsanity.com/paper/2203.05598/) (2022)
- [A Comprehensive Survey of Natural Language Generation Advances from the Perspective of Digital Deception](https://mlsanity.com/paper/2208.05757/) (2022)
- [Exploring the effectiveness of ChatGPT-based feedback compared with teacher feedback and self-feedback: Evidence from Chinese to English translation](https://mlsanity.com/paper/2309.01645/) (2023)
- [Large Language Models for Biomedical Text Simplification: Promising But Not There Yet](https://mlsanity.com/paper/2408.03871/) (2024)
- [Can We Generate Shellcodes via Natural Language? An Empirical Study](https://mlsanity.com/paper/2202.03755/) (2022)
- [HilMeMe: A Human-in-the-Loop Machine Translation Evaluation Metric Looking into Multi-Word Expressions](https://mlsanity.com/paper/2211.05201/) (2022)

## Abstract (arXiv)

To facilitate effective translation modeling and translation studies, one of the crucial questions to address is how to assess translation quality. From the perspectives of accuracy, reliability, repeatability and cost, translation quality assessment (TQA) itself is a rich and challenging task. In this work, we present a high-level and concise survey of TQA methods, including both manual judgement criteria and automated evaluation metrics, which we classify into further detailed sub-categories. We hope that this work will be an asset for both translation model researchers and quality assessment researchers. In addition, we hope that it will enable practitioners to quickly develop a better understanding of the conventional TQA field, and to find corresponding closely relevant evaluation solutions for their own needs. This work may also serve inspire further development of quality assessment and evaluation methodologies for other natural language processing (NLP) tasks in addition to machine translation (MT), such as automatic text summarization (ATS), natural language understanding (NLU) and natural language generation (NLG).