mlsanityData to 14 Oct 2026

M-IFEval: Multilingual Instruction-Following Evaluation

arXiv 2502.04688 ↗First posted 7 Feb 2025cs.CL cs.AIcited by 17

Not on the map. Why?

Abstract (arXiv)

Instruction following is a core capability of modern Large language models (LLMs), making evaluating this capability essential to understanding these models. The Instruction Following Evaluation (IFEval) benchmark from the literature does this using objective criteria, offering a measure of LLM performance without subjective AI or human judgement. However, it only includes English instructions, limiting its ability to assess LLMs in other languages. We propose the Multilingual Instruction Following Evaluation (M-IFEval) benchmark, expanding the evaluation to French, Japanese, and Spanish, with both general and language-specific instructions. Applying this benchmark to 8 state-of-the-art LLMs, we find that benchmark performance across languages and instruction types can vary widely, underscoring the importance of a multilingual benchmark for evaluating LLMs in a diverse cultural context.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 26 of its 38 references that are in the map. First posted on arXiv 7 Feb 2025.

Builds on

26 of 38 references.

References · 26

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  8. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  9. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  10. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  11. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  12. 2018XNLI: Evaluating Cross-lingual Sentence Representationscited by 915
14 more
  1. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  2. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  3. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 509
  4. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500
  5. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 295
  6. 2021Adapting Language Models for Zero-shot Learning by Meta-tuning on Dataset and Prompt Collectionscited by 128
  7. 2023Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queriescited by 82
  8. 2024Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoningcited by 69
  9. 2023PolyLM: An Open Source Polyglot Large Language Modelcited by 64
  10. 2023MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Taskscited by 57
  11. 2023ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferencescited by 43
  12. 2024Leveraging Large Language Models for Learning Complex Legal Concepts through Storytellingcited by 23
  13. 2023Transfer to a Low-Resource Language via Close Relatives: The Case Study on Faroesecited by 15
  14. 2024To Generate or to Retrieve? On the Effectiveness of Artificial Contexts for Medical Open-Domain Question Answeringcited by 15

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 5.

17 papers.

Sort:

By area:

  1. 2025Incentivizing Reasoning for Advanced Instruction-Following of Large Language Modelscited by 17
  2. 2025ReasonIF: Large Reasoning Models Fail to Follow Instructions During Reasoningcited by 13
  3. 2025XIFBench: Evaluating Large Language Models on Multilingual Instruction Followingcited by 11
  4. 2025Inverse IFEval: Can LLMs Unlearn Stubborn Training Conventions to Follow Real Instructions?cited by 10
  5. 2025MultiBanana: A Challenging Benchmark for Multi-Reference Text-to-Image Generationcited by 9
  6. 2026SEQUOR: A Multi-Turn Benchmark for Realistic Constraint Followingcited by 6
  7. 2025Towards Alignment-Centric Paradigm: A Survey of Instruction Tuning in Large Language Modelscited by 5
  8. 2026Neuro-Symbolic Verification on Instruction Following of LLMscited by 4
  9. 2025CAPO: Confidence Aware Preference Optimization Learning for Multilingual Preferencescited by 3
  10. 2025PLaMo 2 Technical Reportcited by 2
  11. 2025Multi-lingual Functional Evaluation for Large Language Modelscited by 1
  12. 2025Instruction-Following Evaluation in Function Calling for Large Language Modelscited by 1
5 more
  1. 2026IndicIFEval: A Benchmark for Verifiable Instruction-Following Evaluation in 14 Indic Languagescited by 1
  2. 2026Disentangling Language Roles in Multilingual LLM Task Executioncited by 1
  3. 2026The Curse of Helpfulness: Inverse Scaling Law in Robustness to Distractor Instructions via DistractionIFcited by 1
  4. 2025Intent-Driven LLM Ensemble Planning for Flexible Multi-Robot Disassembly: Demonstration on EV Batteriescited by 0
  5. 2025CodeAlignBench: Assessing Code Generation Models on Developer-Preferred Code Adjustmentscited by 0