mlsanityData to 7 Oct 2026

AutoFlow: Automated Workflow Generation for Large Language Model Agents

arXiv 2407.12821 ↗First posted 1 Jul 2024cs.CL cs.AI cs.LGcited by 76

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Recent advancements in Large Language Models (LLMs) have shown significant progress in understanding complex natural language. One important application of LLM is LLM-based AI Agent, which leverages the ability of LLM as well as external tools for complex-task solving. To make sure LLM Agents follow an effective and reliable procedure to solve the given task, manually designed workflows are usually used to guide the working mechanism of agents. However, manually designing the workflows requires considerable efforts and domain knowledge, making it difficult to develop and deploy agents on massive scales. To address these issues, we propose AutoFlow, a framework designed to automatically generate workflows for agents to solve complex tasks. AutoFlow takes natural language program as the format of agent workflow and employs a workflow optimization procedure to iteratively optimize the workflow quality. Besides, this work offers two workflow generation methods: fine-tuning-based and in-context-based methods, making the AutoFlow framework applicable to both open-source and closed-source LLMs. Experimental results show that our framework can produce robust and reliable agent workflows. We believe that the automatic generation and interpretation of workflows in natural language represent a promising paradigm for solving complex tasks, particularly with the rapid development of LLMs. The source code of this work is available at https://github.com/agiresearch/AutoFlow.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 49 references that are in the map. First posted on arXiv 1 Jul 2024.

Builds on

32 of 49 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug

Other references · 31

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  7. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  8. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  9. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  10. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  11. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  12. 2018Regularized Evolution for Image Classifier Architecture Searchcited by 902
19 more
  1. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  2. 2023Reasoning with Language Model is Planning with World Modelcited by 846
  3. 2018Efficient Neural Architecture Search via Parameter Sharingcited by 831
  4. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  5. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  6. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
  7. 2020Visual Transformers: Token-based Image Representation and Processing for Computer Visioncited by 333
  8. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  9. 2023OpenAGI: When LLM Meets Domain Expertscited by 196
  10. 2024StateFlow: Enhancing LLM Task-Solving through State-Driven Workflowscited by 65
  11. 2017Simple And Efficient Architecture Search for Convolutional Neural Networkscited by 40
  12. 2024Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agentscited by 39
  13. 2023LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystemcited by 35
  14. 2024FlowMind: Automatic Workflow Generation with LLMscited by 30
  15. 2023Flows: Building Blocks of Reasoning and Collaborating AIcited by 25
  16. 2018GNAS: A Greedy Neural Architecture Search Method for Multi-Attribute Learningcited by 17
  17. 2023GPT is becoming a Turing machine: Here are some ways to program itcited by 16
  18. 2020VEST: Automatic Feature Engineering for Forecastingcited by 14
  19. 2022AutoLossGen: Automatic Loss Function Generation for Recommender Systemscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 37; 2026: 30.

76 papers.

Sort:

By area:

  1. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 405
  2. 2024AFlow: Automating Agentic Workflow Generationcited by 395
  3. 2024Automated Design of Agentic Systemscited by 360
  4. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  5. 2024AgentSquare: Automatic LLM Agent Search in Modular Design Spacecited by 141
  6. 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 106
  7. 2025FlowReasoner: Reinforcing Query-Level Meta-Agentscited by 56
  8. 2026Natural-Language Agent Harnessescited by 54
  9. 2025FLEX: Continuous Agent Evolution via Forward Learning from Experiencecited by 51
  10. 2025ScoreFlow: Mastering LLM Agent Workflows via Score-based Preference Optimizationcited by 50
  11. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  12. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 47
38 more
  1. 2025EvoAgentX: An Automated Framework for Evolving Agentic Workflowscited by 43
  2. 2024Benchmarking Agentic Workflow Generationcited by 36
  3. 2024ComfyBench: Benchmarking LLM-based Agents in ComfyUI for Autonomously Designing Collaborative AI Systemscited by 30
  4. 2024WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Modelscited by 28
  5. 2026From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agentscited by 22
  6. 2026From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flawscited by 21
  7. 2025Weak-for-Strong: Training Weak Meta-Agent to Harness Strong Executorscited by 20
  8. 2025AI Scientists Fail Without Strong Implementation Capabilitycited by 20
  9. 2025RobustFlow: Towards Robust Agentic Workflow Generationcited by 19
  10. 2025Failure-Driven Workflow Refinementcited by 19
  11. 2025A Survey on Agentic Multimodal Large Language Modelscited by 19
  12. 2025SwarmAgentic: Towards Fully Automated Agentic System Generation via Swarm Intelligencecited by 17
  13. 2025A Survey on Agent Workflow -- Status and Futurecited by 14
  14. 2026MemoHarness: Agent Harnesses That Learn from Experiencecited by 14
  15. 2026Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectoriescited by 13
  16. 2025DebFlow: Automating Agent Creation via Agent Debatecited by 10
  17. 2025Sherlock: Reliable and Efficient Agentic Workflow Executioncited by 10
  18. 2026Agentic Environment Engineering for Large Language Models: A Survey of Environment Modeling, Synthesis, Evaluation, and Applicationcited by 8
  19. 2025GeoFlow: Agentic Workflow Automation for Geospatial Taskscited by 7
  20. 2026WirelessAgent++: Automated Agentic Workflow Design and Benchmarking for Wireless Networkscited by 7
  21. 2024MineAgent: Towards Remote-Sensing Mineral Exploration with Multimodal Large Language Modelscited by 6
  22. 2025Fine-Tune an SLM or Prompt an LLM? The Case of Generating Low-Code Workflowscited by 6
  23. 2025PowerChain: A Verifiable Agentic AI System for Automating Distribution Grid Analysescited by 6
  24. 2026HELIX: Model-Harness Co-evolution for Recursive Self-Improvementcited by 6
  25. 2025Inefficiencies of Meta Agents for Agent Designcited by 5
  26. 2025Cache Mechanism for Agent RAG Systemscited by 5
  27. 2026A-MapReduce: Executing Wide Search via Agentic MapReducecited by 5
  28. 2025Compound AI Systems Optimization: A Survey of Methods, Challenges, and Future Directionscited by 4
  29. 2025AdaptFlow: Adaptive Workflow Optimization via Meta-Learningcited by 4
  30. 2025Experience-Guided Adaptation of Inference-Time Reasoning Strategiescited by 3
  31. 2026GraphFlow: A Graph-Based Workflow Management for Efficient LLM-Agent Servingcited by 3
  32. 2026BayesFlow: A Probability Inference Framework for Meta-Agent Assisted Workflow Generationcited by 2
  33. 2026Exploration and Exploitation Errors Are Measurable for Language Model Agentscited by 2
  34. 2026Small Model as Master Orchestrator: Learning Unified Agent-Tool Orchestration with Parallel Subtask Decompositioncited by 2
  35. 2026Chat2Workflow: A Benchmark for Generating Executable Visual Workflows with Natural Languagecited by 2
  36. 2024Action Engine: Automatic Workflow Generation in FaaScited by 1
  37. 2024Bel Esprit: Multi-Agent Framework for Building AI Model Pipelinescited by 1
  38. 2025AIAP: A No-Code Workflow Builder for Non-Experts with Natural Language and Multi-Agent Collaborationcited by 1

50 most cited of 76 All 76 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Characterizing Large Language Model Agentic Workflows: A Study on N8n Ecosystem
  2. 2025SEW: Self-Evolving Agentic Workflows for Automated Code Generation
  3. 2026AgentFactory: Towards Automated Agentic System Design and Optimization
  4. 2024Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights
  5. 2025Scaling LLM Planning: NL2FLOW for Parametric Problem Generation and Rigorous Evaluation
  6. 2025A2Flow: Automating Agentic Workflow Generation via Self-Adaptive Abstraction Operators
  7. 2026FlowBot: Inducing LLM Workflows with Bilevel Optimization and Textual Gradients
  8. 2024ChainBuddy: An AI Agent System for Generating LLM Pipelines