Natural-Language Workflows Are Not Software Yet: Artifact-Driven Compilation for Reliable Agent Execution
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Natural-language workflows offer a software-like interface for agents: domain experts can write reusable procedures, and agents can execute them as instructions. This promise is not yet reliable. Workflow descriptions often leave data dependencies implicit, so the executor must infer which prior results a step should use; agents can also fail to follow long or branching instructions under context pressure. We propose Artic, an artifact-driven workflow compiler that transforms a natural-language workflow into an artifact-driven workflow in which each step declares the artifacts it reads and writes, constraints gate produced artifacts, and explicit control transfers route execution. This representation exposes the enforcement burden placed on agent execution, allowing the compiler to identify steps that depend on too much state or contain difficult control logic and refine them through constrained optimization. To validate the LLM-assisted transformation, Artic decomposes faithfulness checking into local obligations and uses scenario-based dry runs to test whether compiled workflow regions conform to the source workflow. We evaluate Artic on 488 problem instances from 11 real-world domain workflows; it improves task resolve rate by 28 percentage points over the original text workflow. We also show that workflows compiled by Artic are 32 and 56 percentage points more consistent in cross-model and repeated-execution setups, respectively.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 41 of its 64 references that are in the map. First posted on arXiv 21 Aug 2026.
Builds on
41 of 64 references.
Closest ancestors · 5
Same direction, newest first
- 2026SpecOps: A Fully Automated AI Agent Testing Framework in Real-World GUI Environments10 Mar
- 2025WorkTeam: Constructing Workflows from Natural Language with Multi-Agents28 Mar
- 2025FlowAgent: Achieving Compliance and Flexibility for Workflow Agents20 Feb
- 2024WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models8 Nov
- 2024AutoFlow: Automated Workflow Generation for Large Language Model Agents1 Jul
Other references · 36
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Let's Verify Step by Stepcited by 3,789
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
- 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
- 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 653
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 223
24 more
- 2023FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Modelscited by 117
- 2024TrustAgent: Towards Safe and Trustworthy LLM-based Agentscited by 64
- 2023nl2spec: Interactively Translating Unstructured Natural Language to Temporal Logics with Large Language Modelscited by 60
- 2024FlowBench: Revisiting and Benchmarking Workflow-Guided Planning for LLM-based Agentscited by 40
- 2026SkillGen: Verified Inference-Time Agent Skill Synthesiscited by 36
- 2026When Single-Agent with Skills Replace Multi-Agent Systems and When They Failcited by 35
- 2025Enforcing Temporal Constraints for LLM Agentscited by 34
- 2024LIFBench: Evaluating the Instruction Following Performance and Stability of Large Language Models in Long-Context Scenarioscited by 33
- 2026SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?cited by 30
- 2025SOP-Bench: Complex Industrial SOPs for Evaluating LLM Agentscited by 23
- 2024Prompts Are Programs Too! Understanding How Developers Build Software Containing Promptscited by 19
- 2025TAI3: Testing Agent Integrity in Interpreting User Intentcited by 9
- 2026AgentIF-OneDay: A Task-level Instruction-Following Benchmark for General AI Agents in Daily Scenarioscited by 9
- 2026SkCC: Portable and Secure Skill Compilation for Cross-Framework LLM Agentscited by 8
- 2026CHI-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows?cited by 8
- 2024Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMwarecited by 6
- 2025Promptware Engineering: Software Engineering for Prompt-Enabled Systemscited by 6
- 2026LLM Agents in Law: Taxonomy, Applications, and Challengescited by 5
- 2026SkillSmith: Compiling Agent Skills into Boundary-Guided Runtime Interfacescited by 5
- 2026From Plan to Action: How Well Do Agents Follow the Plan?cited by 4
- 2026MASFactory: A Graph-centric Framework for Orchestrating LLM-Based Multi-Agent Systems with Vibe Graphingcited by 3
- 2026LLM Agents as Social Scientists: A Human-AI Collaborative Platform for Social Science Automationcited by 3
- 2026Chat2Workflow: A Benchmark for Generating Executable Visual Workflows with Natural Languagecited by 3
- 2026Enhancing LLM Instruction Following: An Evaluation-Driven Multi-Agentic Workflow for Prompt Instructions Optimizationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026COVENANT: Natural-Language Workflow Compilation for Aligned Agent Execution
- 2026Specifying and Maintaining Agentic Workflows: An Empirical Study of GitHub Agentic Workflows
- 2026Complete Cyclic Subtask Graphs for Tool-Using LLM Agents: Flexibility, Cost, and Bottlenecks in Long-Horizon Workflows
- 2026Learning to Compose for Cross-domain Agentic Workflow Generation
- 2026Learning Compositional Meta-Routing for Agentic Workflows: An Executable Benchmark
- 2026SKILL.nb: Selective Formalization and Gated Execution for Durable Agent Workflows
- 2026From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agents
- 2026Workflow-Localized Mechanism Learning: Attribution-Guided Repair and Knowledge Reuse for Structured Agent Skills