# Large Language Models for Persian  leftrightarrow  English Idiom Translation

arXiv: https://arxiv.org/abs/2412.09993 · HTML: https://mlsanity.com/paper/2412.09993/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 13 Dec 2024.

## Builds on (most cited first, 16 of 16)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing](https://mlsanity.com/paper/2406.08464/) (2024)
- [MADLAD-400: A Multilingual And Document-Level Large Audited Dataset](https://mlsanity.com/paper/2309.04662/) (2023)
- [Testing the Ability of Language Models to Interpret Figurative Language](https://mlsanity.com/paper/2204.12632/) (2022)
- [Multi-lingual and Multi-cultural Figurative Language Understanding](https://mlsanity.com/paper/2305.16171/) (2023)
- [Do GPTs Produce Less Literal Translations?](https://mlsanity.com/paper/2305.16806/) (2023)
- [EPIE Dataset: A Corpus For Possible Idiomatic Expressions](https://mlsanity.com/paper/2006.09479/) (2020)
- [Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language Models](https://mlsanity.com/paper/2308.13961/) (2023)
- [The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilities](https://mlsanity.com/paper/2405.20089/) (2024)
- [MIZAN: A Large Persian-English Parallel Corpus](https://mlsanity.com/paper/1801.02107/) (2018)
- [Optimizing Machine Translation through Prompt Engineering: An Investigation into ChatGPT's Customizability](https://mlsanity.com/paper/2308.01391/) (2023)
- [Creative and Context-Aware Translation of East Asian Idioms with GPT-4](https://mlsanity.com/paper/2410.00988/) (2024)
- [Extracting an English-Persian Parallel Corpus from Comparable Corpora](https://mlsanity.com/paper/1711.00681/) (2017)

## Abstract (arXiv)

Large language models (LLMs) have shown superior capabilities in translating figurative language compared to neural machine translation (NMT) systems. However, the impact of different prompting methods and LLM-NMT combinations on idiom translation has yet to be thoroughly investigated. This paper introduces two parallel datasets of sentences containing idiomatic expressions for PersianrightarrowEnglish and EnglishrightarrowPersian translations, with Persian idioms sampled from our PersianIdioms resource, a collection of 2,200 idioms and their meanings, with 700 including usage examples. Using these datasets, we evaluate various open- and closed-source LLMs, NMT models, and their combinations. Translation quality is assessed through idiom translation accuracy and fluency. We also find that automatic evaluation methods like LLM-as-a-judge, BLEU, and BERTScore are effective for comparing different aspects of model performance. Our experiments reveal that Claude-3.5-Sonnet delivers outstanding results in both translation directions. For EnglishrightarrowPersian, combining weaker LLMs with Google Translate improves results, while PersianrightarrowEnglish translations benefit from single prompts for simpler models and complex prompts for advanced ones.