# M-IFEval: Multilingual Instruction-Following Evaluation

arXiv: https://arxiv.org/abs/2502.04688 · HTML: https://mlsanity.com/paper/2502.04688/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 38 references that are in the map.
- First posted on arXiv 7 Feb 2025.

## Builds on (most cited first, 26 of 26)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://mlsanity.com/paper/2404.04475/) (2024)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [ByT5: Towards a token-free future with pre-trained byte-to-byte models](https://mlsanity.com/paper/2105.13626/) (2021)
- [From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline](https://mlsanity.com/paper/2406.11939/) (2024)
- [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://mlsanity.com/paper/2402.07827/) (2024)
- [Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collections](https://mlsanity.com/paper/2104.04670/) (2021)
- [Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries](https://mlsanity.com/paper/2310.13132/) (2023)
- [Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoning](https://mlsanity.com/paper/2403.20046/) (2024)
- [PolyLM: An Open Source Polyglot Large Language Model](https://mlsanity.com/paper/2307.06018/) (2023)
- [MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks](https://mlsanity.com/paper/2311.07463/) (2023)
- [ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences](https://mlsanity.com/paper/2311.06025/) (2023)
- [Leveraging Large Language Models for Learning Complex Legal Concepts through Storytelling](https://mlsanity.com/paper/2402.17019/) (2024)
- [Transfer to a Low-Resource Language via Close Relatives: The Case Study on Faroese](https://mlsanity.com/paper/2304.08823/) (2023)
- [To Generate or to Retrieve? On the Effectiveness of Artificial Contexts for Medical Open-Domain Question Answering](https://mlsanity.com/paper/2403.01924/) (2024)

## Built on by (most cited first, 17 of 17)

- [Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models](https://mlsanity.com/paper/2506.01413/) (2025)
- [ReasonIF: Large Reasoning Models Fail to Follow Instructions During Reasoning](https://mlsanity.com/paper/2510.15211/) (2025)
- [XIFBench: Evaluating Large Language Models on Multilingual Instruction Following](https://mlsanity.com/paper/2503.07539/) (2025)
- [Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?](https://mlsanity.com/paper/2509.04292/) (2025)
- [MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generation](https://mlsanity.com/paper/2511.22989/) (2025)
- [SEQUOR: A Multi-Turn Benchmark for Realistic Constraint Following](https://mlsanity.com/paper/2605.06353/) (2026)
- [Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Models](https://mlsanity.com/paper/2508.17184/) (2025)
- [Neuro-Symbolic Verification on Instruction Following of LLMs](https://mlsanity.com/paper/2601.17789/) (2026)
- [CAPO: Confidence Aware Preference Optimization Learning for Multilingual Preferences](https://mlsanity.com/paper/2511.07691/) (2025)
- [PLaMo 2 Technical Report](https://mlsanity.com/paper/2509.04897/) (2025)
- [Multi-lingual Functional Evaluation for Large Language Models](https://mlsanity.com/paper/2506.20793/) (2025)
- [Instruction-Following Evaluation in Function Calling for Large Language Models](https://mlsanity.com/paper/2509.18420/) (2025)
- [IndicIFEval: A Benchmark for Verifiable Instruction-Following Evaluation in 14 Indic Languages](https://mlsanity.com/paper/2602.22125/) (2026)
- [Disentangling Language Roles in Multilingual LLM Task Execution](https://mlsanity.com/paper/2605.27649/) (2026)
- [The Curse of Helpfulness: Inverse Scaling Law in Robustness to Distractor Instructions via DistractionIF](https://mlsanity.com/paper/2605.29491/) (2026)
- [Intent-Driven LLM Ensemble Planning for Flexible Multi-Robot Disassembly: Demonstration on EV Batteries](https://mlsanity.com/paper/2510.17576/) (2025)
- [CodeAlignBench: Assessing Code Generation Models on Developer-Preferred Code Adjustments](https://mlsanity.com/paper/2510.27565/) (2025)

## Abstract (arXiv)

Instruction following is a core capability of modern Large language models (LLMs), making evaluating this capability essential to understanding these models. The Instruction Following Evaluation (IFEval) benchmark from the literature does this using objective criteria, offering a measure of LLM performance without subjective AI or human judgement. However, it only includes English instructions, limiting its ability to assess LLMs in other languages. We propose the Multilingual Instruction Following Evaluation (M-IFEval) benchmark, expanding the evaluation to French, Japanese, and Spanish, with both general and language-specific instructions. Applying this benchmark to 8 state-of-the-art LLMs, we find that benchmark performance across languages and instruction types can vary widely, underscoring the importance of a multilingual benchmark for evaluating LLMs in a diverse cultural context.