# MMTE: Corpus and Metrics for Evaluating Machine Translation Quality of Metaphorical Language

arXiv: https://arxiv.org/abs/2406.13698 · HTML: https://mlsanity.com/paper/2406.13698/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 74 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Builds on (most cited first, 17 of 17)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models](https://mlsanity.com/paper/2303.13809/) (2023)
- [TransQuest: Translation Quality Estimation with Cross-lingual Transformers](https://mlsanity.com/paper/2011.01536/) (2020)
- [MERMAID: Metaphor Generation with Symbolism and Discriminative Decoding](https://mlsanity.com/paper/2103.06779/) (2021)
- [Metaphor Generation with Conceptual Mappings](https://mlsanity.com/paper/2106.01228/) (2021)
- [Metaphors in Pre-Trained Language Models: Probing and Generalization Across Datasets and Languages](https://mlsanity.com/paper/2203.14139/) (2022)
- [Bridging Linguistic Typology and Multilingual Machine Translation with Multi-View Language Representations](https://mlsanity.com/paper/2004.14923/) (2020)
- [FrameBERT: Conceptual Metaphor Detection with Frame Embedding Learning](https://mlsanity.com/paper/2302.04834/) (2023)
- [Metaphor Detection with Effective Context Denoising](https://mlsanity.com/paper/2302.05611/) (2023)
- [As Little as Possible, as Much as Necessary: Detecting Over- and Undertranslations with Contrastive Conditioning](https://mlsanity.com/paper/2203.01927/) (2022)
- [The Iron(ic) Melting Pot: Reviewing Human Evaluation in Humour, Irony and Sarcasm Generation](https://mlsanity.com/paper/2311.05552/) (2023)
- [CMDAG: A Chinese Metaphor Dataset with Annotated Grounds as CoT for Boosting Metaphor Generation](https://mlsanity.com/paper/2402.13145/) (2024)
- [From Handcrafted Features to LLMs: A Brief Survey for Machine Translation Quality Estimation](https://mlsanity.com/paper/2403.14118/) (2024)
- ["Love is as Complex as Math": Metaphor Generation System for Social Chatbot](https://mlsanity.com/paper/2001.00733/) (2020)
- [The Secret of Metaphor on Expressing Stronger Emotion](https://mlsanity.com/paper/2301.13042/) (2023)
- [Zero-Shot Translation Quality Estimation with Explicit Cross-Lingual Patterns](https://mlsanity.com/paper/2010.04989/) (2020)

## Built on by (most cited first, 10 of 10)

- [Overview of the NLPCC 2024 Shared Task on Chinese Metaphor Generation](https://mlsanity.com/paper/2408.04378/) (2024)
- [MetaHOPE: A Metaphor-Oriented Evaluation Framework for Analysing MT and LLM Translation Errors](https://mlsanity.com/paper/2607.00848/) (2026)
- [Pun Intended: Multi-Agent Translation of Wordplay with Contrastive Learning and Phonetic-Semantic Embeddings for CLEF JOKER 2025 Task 2](https://mlsanity.com/paper/2507.06506/) (2025)
- [Last Translation Benchmark](https://mlsanity.com/paper/2609.04173/) (2026)
- [Natural Language Generation](https://mlsanity.com/paper/2503.16728/) (2025)
- [Towards High-Level Semantic Intelligence](https://mlsanity.com/paper/2607.24082/) (2026)
- [Towards End-to-End Multilingual Metaphor Processing: Integrating Detection, Translation, and Evaluation](https://mlsanity.com/paper/2608.04260/) (2026)
- [Searching for Sound-Meaning Collisions: Graph-Based Affordance Retrieval and Multi-Evaluator Ranking for Pun Translation at CLEF 2026 JOKER Task 2](https://mlsanity.com/paper/2608.04299/) (2026)
- [Pun Intended: Multi-Agent Translation of Wordplay with Contrastive Learning and Phonetic-Semantic Embeddings](https://mlsanity.com/paper/2608.04311/) (2026)
- [AraMIP: Extending MIPVU Towards Metaphor Identification in Arabic](https://mlsanity.com/paper/2609.17235/) (2026)

## Abstract (arXiv)

Machine Translation (MT) has developed rapidly since the release of Large Language Models and current MT evaluation is performed through comparison with reference human translations or by predicting quality scores from human-labeled data. However, these mainstream evaluation methods mainly focus on fluency and factual reliability, whilst paying little attention to figurative quality. In this paper, we investigate the figurative quality of MT and propose a set of human evaluation metrics focused on the translation of figurative language. We additionally present a multilingual parallel metaphor corpus generated by post-editing. Our evaluation protocol is designed to estimate four aspects of MT: Metaphorical Equivalence, Emotion, Authenticity, and Quality. In doing so, we observe that translations of figurative expressions display different traits from literal ones.