mlsanityData to 7 Oct 2026

FOFO: A Benchmark to Evaluate LLMs' Format-Following Capability

arXiv 2402.18667 ↗First posted 28 Feb 2024cs.CLcited by 79

Not on the map. Why?

Abstract (arXiv)

This paper presents FoFo, a pioneering benchmark for evaluating large language models' (LLMs) ability to follow complex, domain-specific formats, a crucial yet underexamined capability for their application as AI agents. Despite LLMs' advancements, existing benchmarks fail to assess their format-following proficiency adequately. FoFo fills this gap with a diverse range of real-world formats and instructions, developed through an AI-Human collaborative method. Our evaluation across both open-source (e.g., Llama 2, WizardLM) and closed-source (e.g., GPT-4, PALM2, Gemini) LLMs highlights three key findings: open-source models significantly lag behind closed-source ones in format adherence; LLMs' format-following performance is independent of their content generation quality; and LLMs' format proficiency varies across different domains. These insights suggest the need for specialized tuning for format-following skills and highlight FoFo's role in guiding the selection of domain-specific AI agents. FoFo is released here at https://github.com/SalesforceAIResearch/FoFo.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Builds on 19 of its 35 references that are in the map. First posted on arXiv 28 Feb 2024.

Builds on

19 of 35 references.

References · 19

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  8. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  9. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  10. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  11. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  12. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,116
7 more
  1. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  2. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 250
  3. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 213
  4. 2023Understanding the Benefits and Challenges of Using Large Language Model-based Conversational Agents for Mental Well-being Supportcited by 75
  5. 2023Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrongcited by 51
  6. 2023TrafficSafetyGPT: Tuning a Pre-trained Large Language Model to a Domain-Specific Expert in Transportation Safetycited by 27
  7. 2024Benchmarking Large Language Models on Controllable Generation under Diversified Instructionscited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 36; 2025: 32; 2026: 11.

79 papers.

Sort:

By area:

  1. 2024Benchmarking Complex Instruction-Following with Multiple Constraints Compositioncited by 147
  2. 2024Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Followingcited by 94
  3. 2024Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Modelscited by 90
  4. 2024AgentInstruct: Toward Generative Teaching with Agentic Flowscited by 88
  5. 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 84
  6. 2024Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Modelscited by 73
  7. 2024CFBench: A Comprehensive Constraints-Following Benchmark for LLMscited by 65
  8. 2024Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Modelscited by 53
  9. 2025The Price of Format: Diversity Collapse in LLMscited by 43
  10. 2024CompassJudger-1: All-in-one Judge Model Helps Model Evaluation and Evolutioncited by 42
  11. 2025Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Modelscited by 28
  12. 2024From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Modelscited by 27
38 more
  1. 2025Learning to Generate Structured Output with Schema Reinforcement Learningcited by 27
  2. 2025Speech-IFEval: Evaluating Instruction-Following and Quantifying Catastrophic Forgetting in Speech-Aware Language Modelscited by 27
  3. 2024SysBench: Can Large Language Models Follow System Messages?cited by 26
  4. 2024Do LLMs estimate uncertainty well in instruction-following?cited by 26
  5. 2024RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarioscited by 26
  6. 2024Do LLMs "know" internally when they follow instructions?cited by 25
  7. 2024LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMscited by 24
  8. 2024SPaR: Self-Play with Tree-Search Refinement to Improve Instruction-Following in Large Language Modelscited by 23
  9. 2025StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputscited by 23
  10. 2025CiteEval: Principle-Driven Citation Evaluation for Source Attributioncited by 19
  11. 2025SLOT: Structuring the Output of Large Language Modelscited by 18
  12. 2025Spotlight Your Instructions: Instruction-following with Dynamic Attention Steeringcited by 18
  13. 2024Large Language Models as Software Components: A Taxonomy for LLM-Integrated Applicationscited by 17
  14. 2025Condor: Enhance LLM Alignment with Knowledge-Driven Data Synthesis and Refinementcited by 17
  15. 2025Incentivizing Reasoning for Advanced Instruction-Following of Large Language Modelscited by 17
  16. 2025Verifiable Format Control for Large Language Model Generationscited by 16
  17. 2025GuideBench: Benchmarking Domain-Oriented Guideline Following for LLM Agentscited by 14
  18. 2025Pre-Trained Policy Discriminators are General Reward Modelscited by 14
  19. 2024MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Casescited by 13
  20. 2025Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judgecited by 13
  21. 2025ReasonIF: Large Reasoning Models Fail to Follow Instructions During Reasoningcited by 12
  22. 2024RuleR: Improving LLM Controllability by Rule-based Data Recyclingcited by 11
  23. 2025EvaLearn: Quantifying the Learning Capability and Efficiency of LLMs via Sequential Problem Solvingcited by 10
  24. 2024Query-OPT: Optimizing Inference of Large Language Models via Multi-Query Instructions in Meeting Summarizationcited by 9
  25. 2024Toward General Instruction-Following Alignment for Retrieval-Augmented Generationcited by 9
  26. 2024ReFF: Reinforcing Format Faithfulness in Language Models across Varied Taskscited by 9
  27. 2025EIFBENCH: Extremely Complex Instruction Following Benchmark for Large Language Modelscited by 9
  28. 2024DOLOMITES: Domain-Specific Long-Form Methodical Taskscited by 8
  29. 2024MMMT-IF: A Challenging Multimodal Multi-Turn Instruction Following Benchmarkcited by 8
  30. 2024A Systematic Review on Prompt Engineering in Large Language Models for K-12 STEM Educationcited by 8
  31. 2026AgentIF-OneDay: A Task-level Instruction-Following Benchmark for General AI Agents in Daily Scenarioscited by 8
  32. 2024Revolutionizing Database Q&A with Large Language Models: Comprehensive Benchmark and Evaluationcited by 7
  33. 2025Enabling Autoregressive Models to Fill In Masked Tokenscited by 7
  34. 2026Empowering Reliable Visual-Centric Instruction Following in MLLMscited by 7
  35. 2024BenchAgents: Multi-Agent Systems for Structured Benchmark Creationcited by 6
  36. 2024Evalverse: Unified and Accessible Library for Large Language Model Evaluationcited by 5
  37. 2026SEQUOR: A Multi-Turn Benchmark for Realistic Constraint Followingcited by 5
  38. 2024Improving the Validity and Practical Usefulness of AI/ML Evaluations Using an Estimands Frameworkcited by 4

50 most cited of 79 All 79 (JSON)