mlsanityData to 7 Oct 2026

FlowBench: Revisiting and Benchmarking Workflow-Guided Planning for LLM-based Agents

arXiv 2406.14884 ↗First posted 21 Jun 2024cs.CLcited by 39

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

LLM-based agents have emerged as promising tools, which are crafted to fulfill complex tasks by iterative planning and action. However, these agents are susceptible to undesired planning hallucinations when lacking specific knowledge for expertise-intensive tasks. To address this, preliminary attempts are made to enhance planning reliability by incorporating external workflow-related knowledge. Despite the promise, such infused knowledge is mostly disorganized and diverse in formats, lacking rigorous formalization and comprehensive comparisons. Motivated by this, we formalize different formats of workflow knowledge and present FlowBench, the first benchmark for workflow-guided planning. FlowBench covers 51 different scenarios from 6 domains, with knowledge presented in diverse formats. To assess different LLMs on FlowBench, we design a multi-tiered evaluation framework. We evaluate the efficacy of workflow knowledge across multiple formats, and the results indicate that current LLM agents need considerable improvements for satisfactory planning. We hope that our challenging benchmark can pave the way for future agent planning research.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 30 of its 45 references that are in the map. First posted on arXiv 21 Jun 2024.

Builds on

30 of 45 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agents2 Feb
  2. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents24 Jan
  3. 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents1 Jan
  4. 2023AutoAgents: A Framework for Automatic Agent Generation29 Sep
  5. 2023A Survey on Large Language Model based Autonomous Agents22 Aug
  6. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug

Other references · 24

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  5. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  7. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
  8. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  9. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 786
  10. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 372
  11. 2023HuatuoGPT, towards Taming Language Model to Be a Doctorcited by 211
  12. 2018WikiHow: A Large Scale Text Summarization Datasetcited by 203
12 more
  1. 2023SmartPlay: A Benchmark for LLMs as Intelligent Agentscited by 123
  2. 2023Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-world Multi-turn Dialoguecited by 118
  3. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  4. 2023T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Stepcited by 91
  5. 2023Benchmarking Large Language Models on CMExam -- A Comprehensive Chinese Medical Exam Datasetcited by 90
  6. 2024KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graphcited by 90
  7. 2024Generate-on-Graph: Treat LLM as both Agent and KG in Incomplete Knowledge Graph Question Answeringcited by 59
  8. 2020STAR: A Schema-Guided Dialog Dataset for Transfer Learningcited by 43
  9. 2023Knowledge-Augmented Large Language Models for Personalized Contextual Query Suggestioncited by 43
  10. 2023ProAgent: From Robotic Process Automation to Agentic Process Automationcited by 26
  11. 2023Knowledge-enhanced Agents for Interactive Text Gamescited by 10
  12. 2024Evaluating the Factuality of Large Language Models using Large-Scale Knowledge Graphscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 18; 2026: 17.

39 papers.

Sort:

By area:

  1. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  2. 2024AI Agents That Mattercited by 199
  3. 2025Survey on Evaluation of LLM-based Agentscited by 190
  4. 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Surveycited by 56
  5. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
  6. 2025Large Language Models for Planning: A Comprehensive and Systematic Surveycited by 38
  7. 2024Benchmarking Agentic Workflow Generationcited by 36
  8. 2026Why Reasoning Fails to Plan: A Planning-Centric Analysis of Long-Horizon Decision Making in LLM Agentscited by 22
  9. 2026From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agentscited by 22
  10. 2025Goal Alignment in LLM-Based User Simulators for Conversational AIcited by 19
  11. 2025SEW: Self-Evolving Agentic Workflows for Automated Code Generationcited by 18
  12. 2025Position: The Real Barrier to LLM Agent Usability is Agentic ROIcited by 12
27 more
  1. 2025Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Surveycited by 12
  2. 2025Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agentscited by 11
  3. 2025A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directionscited by 9
  4. 2025Agentic Knowledgeable Self-awarenesscited by 8
  5. 2024Reflection-Bench: Evaluating Epistemic Agency in Large Language Modelscited by 7
  6. 2025ToolSpectrum : Towards Personalized Tool Utilization for Large Language Modelscited by 6
  7. 2025How Far Are LLMs from Symbolic Planners? An NLP-Based Perspectivecited by 6
  8. 2026AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraintscited by 6
  9. 2026Talk, Evaluate, Diagnose: User-aware Agent Evaluation with Automated Error Analysiscited by 5
  10. 2024From An LLM Swarm To A PDDL-Empowered HIVE: Planning Self-Executed Instructions In A Multi-Modal Junglecited by 4
  11. 2025Scaling LLM Planning: NL2FLOW for Parametric Problem Generation and Rigorous Evaluationcited by 3
  12. 2026Execution-State-Aware LLM Reasoning for Automated Proof-of-Vulnerability Generationcited by 3
  13. 2026The Scaling Laws of Skills in LLM Agent Systemscited by 3
  14. 2026GraphFlow: A Graph-Based Workflow Management for Efficient LLM-Agent Servingcited by 3
  15. 2025AURA: A Diagnostic Framework for Tracking User Satisfaction of Interactive Planning Agentscited by 2
  16. 2025Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agentscited by 2
  17. 2025CoSPlan: Corrective Sequential Planning via Scene Graph Incremental Updatescited by 1
  18. 2026CCR-Bench: A Comprehensive Benchmark for Evaluating LLMs on Complex Constraints, Control Flows, and Real-World Casescited by 1
  19. 2026T1-Bench: Benchmarking Multi-Scenario Agents in Real-World Domainscited by 1
  20. 2025A Multimodal GUI Architecture for Interfacing with LLM-Based Conversational Assistantscited by 0
  21. 2026Forest-Chat: Adapting Vision-Language Agents for Interactive Forest Change Analysiscited by 0
  22. 2026FM SO.P: A Progressive Task Mixture Framework with Automatic Evaluation for Cross-Domain SOP Understandingcited by 0
  23. 2026Humans' ALMANAC: A Human Collaboration Dataset of Action-Level Mental Model Annotations for Agent Collaborationcited by 0
  24. 2026OrchBench: Evaluating Multi-Agent Orchestration Plans in Isolation via Deterministic Simulationcited by 0
  25. 2026Natural-Language Workflows Are Not Software Yet: Artifact-Driven Compilation for Reliable Agent Executioncited by 0
  26. 2026Procedural Graphs: Self-Evolving Execution Structures for LLM Agentscited by 0
  27. 2026DeFiFlowBench: Benchmarking and Improving Safe Executability in Natural-Language DeFi Workflow Synthesiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Agent Planning Benchmark: A Diagnostic Framework for Planning Capabilities in LLM Agents
  2. 2025Procedural Knowledge Improves Agentic LLM Workflows
  3. 2026FlowScout: From Execution Feedback to Reliable Tool-Using Agent Workflows
  4. 2025SagaLLM: Context Management, Validation, and Transaction Guarantees for Multi-Agent LLM Planning
  5. 2025PlanGenLLMs: A Modern Survey of LLM Planning Capabilities
  6. 2026PlanBench-XL: Evaluating Long-Horizon Planning of LLM Tool-Use Agents in Large-Scale Tool Ecosystems
  7. 2026PDEU-Bench: Benchmarking the Personalized Planning Lifecycle of Tool-Calling LLM Agents
  8. 2025Flow: Modularized Agentic Workflow Automation