Characterizing Large Language Model Agentic Workflows: A Study on N8n Ecosystem
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Large Language Models (LLMs) are rapidly being adopted in low-code and no-code automation platforms, where non-expert users design workflows that combine natural language understanding with external services and APIs. LLM agents are LLM systems that use LLMs as a core "brain" to reason, plan, and autonomously execute complex, multi-step tasks. In this paper, we present the first large-scale empirical study of LLM agentic workflows in low-code automation platforms. We analyze more than 6,000 publicly available n8n workflows and examine four aspects of their design: task distribution, structural and tool use patterns, reliability mechanisms, and autonomy levels. Our analysis shows that LLM workflows are not merely prompt response pipelines. Instead, LLMs are commonly embedded within broader automation structures involving control logic, external tools, communication services, storage systems, and human review points. We further find that while many workflows include lightweight post-processing or routing logic after LLM execution, explicit reliability mechanisms such as structured fallback paths, repair loops, failure-specific alerts, and human approval gates remain relatively uncommon. These results reveal a gap between the increasing deployment of LLM agents in practical automation ecosystems and the limited engineering support for reliability, safety, and governance. Overall, our study provides ten empirical findings and five research takeaways for researchers, platform developers, and practitioners seeking to understand and improve real-world LLM agentic workflows.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 31 of its 37 references that are in the map. First posted on arXiv 27 Jun 2026.
Builds on
31 of 37 references.
Closest ancestors · 3
Same direction, newest first
- 2024WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models8 Nov
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug
Other references · 28
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
16 more
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
- 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 574
- 2022ScienceWorld: Is your Agent Smarter than a 5th Grader?cited by 298
- 2022MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoningcited by 153
- 2020A Survey of Knowledge-Enhanced Text Generationcited by 119
- 2023No Need to Lift a Finger Anymore? Assessing the Quality of Code Generation by ChatGPTcited by 45
- 2023ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generationcited by 32
- 2024An Empirical Study on Low-Code Programming using Traditional vs Large Language Model Supportcited by 7
- 2025Artemis: Toward Accurate Detection of Server-Side Request Forgeries through LLM-Assisted Inter-Procedural Path-Sensitive Taint Analysiscited by 5
- 2026MCP-SandboxScan: WASM-based Secure Execution and Runtime Analysis for MCP Toolscited by 3
- 2024LLM-CompDroid: Repairing Configuration Compatibility Bugs in Android Apps with Pre-trained Large Language Modelscited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024AutoFlow: Automated Workflow Generation for Large Language Model Agents
- 2025A Comprehensive Survey on Benchmarks and Solutions in Software Engineering of LLM-Empowered Agentic System
- 2026The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration
- 2024If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents
- 2024Coalitions of Large Language Models Increase the Robustness of AI Agents
- 2024Designing Algorithms Empowered by Language Models: An Analytical Framework, Case Studies, and Insights
- 2026UniACE: A Unified Framework for Evaluating LLM Agentic Capabilities
- 2026Where Agent Frameworks Fall Short: Examining Functional Challenges and Usability Concerns