mlsanityData to 7 Oct 2026

Instruction-Following Evaluation for Large Language Models

arXiv 2311.07911 ↗First posted 14 Nov 2023cs.CL cs.AI cs.LGcited by 1,143

Not on the map. Why?

Cited by 219 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

One core capability of Large Language Models (LLMs) is to follow natural language instructions. However, the evaluation of such abilities is not standardized: Human evaluations are expensive, slow, and not objectively reproducible, while LLM-based auto-evaluation is potentially biased or limited by the ability of the evaluator LLM. To overcome these issues, we introduce Instruction-Following Eval (IFEval) for large language models. IFEval is a straightforward and easy-to-reproduce evaluation benchmark. It focuses on a set of "verifiable instructions" such as "write in more than 400 words" and "mention the keyword of AI at least 3 times". We identified 25 types of those verifiable instructions and constructed around 500 prompts, with each prompt containing one or more verifiable instructions. We show evaluation results of two widely available LLMs on the market. Our code and data can be found at https://github.com/google-research/google-research/tree/master/instruction_following_eval

Key facts. Cited by 1,143 papers in the map (data through 7 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 14 Nov 2023.

Builds on

20 of 27 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  8. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  9. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  10. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,462
  11. 2023A Survey on Evaluation of Large Language Modelscited by 1,227
  12. 2023PaLM 2 Technical Reportcited by 1,154
8 more
  1. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  2. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  3. 2023Instruction Tuning with GPT-4cited by 676
  4. 2023GPTScore: Evaluate as You Desirecited by 357
  5. 2023Evaluating Large Language Models on Controlled Generation Taskscited by 73
  6. 2023Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarizationcited by 70
  7. 2023Large Language Models are Diverse Role-Players for Summarization Evaluationcited by 62
  8. 2023Towards Better Evaluation of Instruction-Following: A Case-Study in Summarizationcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 128; 2025: 417; 2026: 598.

1,143 papers.

Sort:

By area:

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2025Qwen3-VL Technical Reportcited by 2,534
  5. 2024Qwen2 Technical Reportcited by 2,076
  6. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  7. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  8. 2024ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Toolscited by 1,096
  9. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  10. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 988
  11. 2024DeepSeek-V3 Technical Reportcited by 724
  12. 2024Phi-4 Technical Reportcited by 675
38 more
  1. 2024DeepSeek LLM: Scaling Open-Source Language Models with Longtermismcited by 621
  2. 2025Kimi K2: Open Agentic Intelligencecited by 523
  3. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  4. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  5. 2025Qwen3-Omni Technical Reportcited by 451
  6. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 446
  7. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 341
  8. 2024OLMoE: Open Mixture-of-Experts Language Modelscited by 291
  9. 2026Reinforcement Learning via Self-Distillationcited by 285
  10. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  11. 2025Small Language Models are the Future of Agentic AIcited by 254
  12. 2024LiveBench: A Challenging, Contamination-Limited LLM Benchmarkcited by 242
  13. 2024Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimizationcited by 238
  14. 2026Self-Distillation Enables Continual Learningcited by 212
  15. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  16. 2024VoiceBench: Benchmarking LLM-Based Voice Assistantscited by 192
  17. 2024Advancing LLM Reasoning Generalists with Preference Treescited by 191
  18. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 187
  19. 2026Gemma 4 Technical Reportcited by 187
  20. 2025Generalizing Verifiable Instruction Followingcited by 178
  21. 2025Know Me, Respond to Me: Benchmarking LLMs for Dynamic User Profiling and Personalized Responses at Scalecited by 163
  22. 2025RL's Razor: Why Online Reinforcement Learning Forgets Lesscited by 162
  23. 2026On-Policy Context Distillation for Language Modelscited by 161
  24. 2025Qwen2.5-1M Technical Reportcited by 158
  25. 2025LLaDA2.0: Scaling Up Diffusion Language Models to 100Bcited by 154
  26. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 151
  27. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  28. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 147
  29. 2025Phi-4-reasoning Technical Reportcited by 146
  30. 2026Qwen3.5-Omni Technical Reportcited by 146
  31. 2024Nemotron-4 340B Technical Reportcited by 143
  32. 2024Compact Language Models via Pruning and Knowledge Distillationcited by 142
  33. 2025Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMscited by 142
  34. 2025RewardBench 2: Advancing Reward Model Evaluationcited by 136
  35. 2025SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generationcited by 118
  36. 2025OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignmentcited by 118
  37. 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttrainingcited by 117
  38. 2025Checklists Are Better Than Reward Models For Aligning Language Modelscited by 115

50 most cited of 1,143 All 1,143 (JSON)