# D'ej`a Vu: Multilingual LLM Evaluation through the Lens of Machine Translation Evaluation

arXiv: https://arxiv.org/abs/2504.11829 · HTML: https://mlsanity.com/paper/2504.11829/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2025.

## Built on by (most cited first, 3 of 3)

- [UPDESH: Synthesizing Grounded Instruction Tuning Data for 13 Indic Languages](https://mlsanity.com/paper/2509.21294/) (2025)
- [NeoBabel: A Multilingual Open Tower for Visual Generation](https://mlsanity.com/paper/2507.06137/) (2025)
- [Estonian WinoGrande Dataset: Comparative Analysis of LLM Performance on Human and Machine Translation](https://mlsanity.com/paper/2511.17290/) (2025)

## Abstract (arXiv)

Generation capabilities and language coverage of multilingual large language models (mLLMs) are advancing rapidly. However, evaluation practices for generative abilities of mLLMs are still lacking comprehensiveness, scientific rigor, and consistent adoption across research labs, which undermines their potential to meaningfully guide mLLM development. We draw parallels with machine translation (MT) evaluation, a field that faced similar challenges and has, over decades, developed transparent reporting standards and reliable evaluations for multilingual generative models. Through targeted experiments across key stages of the generative evaluation pipeline, we demonstrate how best practices from MT evaluation can deepen the understanding of quality differences between models. Additionally, we identify essential components for robust meta-evaluation of mLLMs, ensuring the evaluation methods themselves are rigorously assessed. We distill these insights into a checklist of actionable recommendations for mLLM research and development.