# From Handcrafted Features to LLMs: A Brief Survey for Machine Translation Quality Estimation

arXiv: https://arxiv.org/abs/2403.14118 · HTML: https://mlsanity.com/paper/2403.14118/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 71 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task](https://mlsanity.com/paper/2209.06243/) (2022)
- [NDC-Scene: Boost Monocular 3D Semantic Scene Completion in Normalized Device Coordinates Space](https://mlsanity.com/paper/2309.14616/) (2023)
- [Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models](https://mlsanity.com/paper/2303.13809/) (2023)
- [TransQuest: Translation Quality Estimation with Cross-lingual Transformers](https://mlsanity.com/paper/2011.01536/) (2020)
- [OpenKiwi: An Open Source Framework for Quality Estimation](https://mlsanity.com/paper/1902.08646/) (2019)
- [MLQE-PE: A Multilingual Quality Estimation and Post-Editing Dataset](https://mlsanity.com/paper/2010.04480/) (2020)
- [Improving Depth Gradient Continuity in Transformers: A Comparative Study on Monocular Depth Estimation with CNN](https://mlsanity.com/paper/2308.08333/) (2023)
- [UniTE: Unified Translation Evaluation](https://mlsanity.com/paper/2204.13346/) (2022)
- [INSTRUCTSCORE: Explainable Text Generation Evaluation with Finegrained Feedback](https://mlsanity.com/paper/2305.14282/) (2023)
- [Toward Human-Like Evaluation for Natural Language Generation with Error Analysis](https://mlsanity.com/paper/2212.10179/) (2022)
- [Unbabel's Participation in the WMT19 Translation Quality Estimation Shared Task](https://mlsanity.com/paper/1907.10352/) (2019)
- ["Bilingual Expert" Can Find Translation Errors](https://mlsanity.com/paper/1807.09433/) (2018)
- [DirectQE: Direct Pretraining for Machine Translation Quality Estimation](https://mlsanity.com/paper/2105.07149/) (2021)
- [Ensembling Factored Neural Machine Translation Models for Automatic Post-Editing and Quality Estimation](https://mlsanity.com/paper/1706.05083/) (2017)
- [Knowledge-Prompted Estimator: A Novel Approach to Explainable Machine Translation Assessment](https://mlsanity.com/paper/2306.07486/) (2023)

## Built on by (most cited first, 10 of 10)

- [The Scales of Justitia: A Comprehensive Survey on Safety Evaluation of LLMs](https://mlsanity.com/paper/2506.11094/) (2025)
- [M-MAD: Multidimensional Multi-Agent Debate for Advanced Machine Translation Evaluation](https://mlsanity.com/paper/2412.20127/) (2024)
- [MMTE: Corpus and Metrics for Evaluating Machine Translation Quality of Metaphorical Language](https://mlsanity.com/paper/2406.13698/) (2024)
- [Speculative Coreset Selection for Task-Specific Fine-tuning](https://mlsanity.com/paper/2410.01296/) (2024)
- [Adapting Large Language Models to Log Analysis with Interpretable Domain Knowledge](https://mlsanity.com/paper/2412.01377/) (2024)
- [Bridging the Linguistic Divide: A Survey on Leveraging Large Language Models for Machine Translation](https://mlsanity.com/paper/2504.01919/) (2025)
- [LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety Inconsistencies](https://mlsanity.com/paper/2412.15035/) (2024)
- [Generalized Correctness Models: Learning Calibrated and Model-Agnostic Correctness Predictors from Historical Patterns](https://mlsanity.com/paper/2509.24988/) (2025)
- [A Rising Tide Lifts All Boats: MTQE Rewards for Idioms Improve General Translation Quality](https://mlsanity.com/paper/2601.06307/) (2026)
- [Model-Based Quality Assessment for Massively Multilingual Parallel Data](https://mlsanity.com/paper/2606.00285/) (2026)

## Abstract (arXiv)

Machine Translation Quality Estimation (MTQE) is the task of estimating the quality of machine-translated text in real time without the need for reference translations, which is of great importance for the development of MT. After two decades of evolution, QE has yielded a wealth of results. This article provides a comprehensive overview of QE datasets, annotation methods, shared tasks, methodologies, challenges, and future research directions. It begins with an introduction to the background and significance of QE, followed by an explanation of the concepts and evaluation metrics for word-level QE, sentence-level QE, document-level QE, and explainable QE. The paper categorizes the methods developed throughout the history of QE into those based on handcrafted features, deep learning, and Large Language Models (LLMs), with a further division of deep learning-based methods into classic deep learning and those incorporating pre-trained language models (LMs). Additionally, the article details the advantages and limitations of each method and offers a straightforward comparison of different approaches. Finally, the paper discusses the current challenges in QE research and provides an outlook on future research directions.