# Comparative Study of Multilingual Idioms and Similes in Large Language Models

arXiv: https://arxiv.org/abs/2410.16461 · HTML: https://mlsanity.com/paper/2410.16461/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Oct 2024.

## Built on by (most cited first, 10 of 10)

- [The State of Multilingual LLM Safety Research: From Measuring the Language Gap to Mitigating It](https://mlsanity.com/paper/2505.24119/) (2025)
- [Memorization or Reasoning? Exploring the Idiom Understanding of LLMs](https://mlsanity.com/paper/2505.16216/) (2025)
- [Isolating Culture Neurons in Multilingual Large Language Models](https://mlsanity.com/paper/2508.02241/) (2025)
- [When Words Don't Mean What They Say: Figurative Understanding in Bengali Idioms](https://mlsanity.com/paper/2602.12921/) (2026)
- [Evaluating the Impact of Verbal Multiword Expressions on Machine Translation](https://mlsanity.com/paper/2508.17458/) (2025)
- [Multilingual Idioms in Sentences and Conversations Across High-, Medium-, and Low-Resource Languages](https://mlsanity.com/paper/2606.02147/) (2026)
- [G-IdiomAlign: A Gloss-Pivoted Benchmark for Cross-Lingual Idiom Alignment](https://mlsanity.com/paper/2606.18989/) (2026)
- [Towards High-Level Semantic Intelligence](https://mlsanity.com/paper/2607.24082/) (2026)
- [Easy to Complete, Hard to Choose: Investigating LLM Performance on the ProverbIT Benchmark](https://mlsanity.com/paper/2608.04670/) (2026)
- [Wisdom in Unity: The Role of Multilingual Training in Figurative Language Identification in Proverbs](https://mlsanity.com/paper/2608.08090/) (2026)

## Abstract (arXiv)

This study addresses the gap in the literature concerning the comparative performance of LLMs in interpreting different types of figurative language across multiple languages. By evaluating LLMs using two multilingual datasets on simile and idiom interpretation, we explore the effectiveness of various prompt engineering strategies, including chain-of-thought, few-shot, and English translation prompts. We extend the language of these datasets to Persian as well by building two new evaluation sets. Our comprehensive assessment involves both closed-source (GPT-3.5, GPT-4o mini, Gemini 1.5), and open-source models (Llama 3.1, Qwen2), highlighting significant differences in performance across languages and figurative types. Our findings reveal that while prompt engineering methods are generally effective, their success varies by figurative type, language, and model. We also observe that open-source models struggle particularly with low-resource languages in similes. Additionally, idiom interpretation is nearing saturation for many languages, necessitating more challenging evaluations.