# Proverbs Run in Pairs: Evaluating Proverb Translation Capability of Large Language Model

arXiv: https://arxiv.org/abs/2501.11953 · HTML: https://mlsanity.com/paper/2501.11953/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jan 2025.

## Built on by (most cited first, 8 of 8)

- [Culture-Aware Machine Translation in Large Language Models: Benchmarking and Investigation](https://mlsanity.com/paper/2604.24361/) (2026)
- [TransEvalnia: Reasoning-based Evaluation and Ranking of Translations](https://mlsanity.com/paper/2507.12724/) (2025)
- [LiveCultureBench: a Multi-Agent, Multi-Cultural Benchmark for Large Language Models in Dynamic Social Simulations](https://mlsanity.com/paper/2603.01952/) (2026)
- [Proverbs or Pythian Oracles? Sentiments and Emotions in Greek Sayings](https://mlsanity.com/paper/2510.13341/) (2025)
- [Evidence-based Distributional Alignment for Large Language Models](https://mlsanity.com/paper/2603.13305/) (2026)
- [Towards High-Level Semantic Intelligence](https://mlsanity.com/paper/2607.24082/) (2026)
- [Easy to Complete, Hard to Choose: Investigating LLM Performance on the ProverbIT Benchmark](https://mlsanity.com/paper/2608.04670/) (2026)
- [Wisdom in Unity: The Role of Multilingual Training in Figurative Language Identification in Proverbs](https://mlsanity.com/paper/2608.08090/) (2026)

## Abstract (arXiv)

Despite achieving remarkable performance, machine translation (MT) research remains underexplored in terms of translating cultural elements in languages, such as idioms, proverbs, and colloquial expressions. This paper investigates the capability of state-of-the-art neural machine translation (NMT) and large language models (LLMs) in translating proverbs, which are deeply rooted in cultural contexts. We construct a translation dataset of standalone proverbs and proverbs in conversation for four language pairs. Our experiments show that the studied models can achieve good translation between languages with similar cultural backgrounds, and LLMs generally outperform NMT models in proverb translation. Furthermore, we find that current automatic evaluation metrics such as BLEU, CHRF++ and COMET are inadequate for reliably assessing the quality of proverb translation, highlighting the need for more culturally aware evaluation metrics.