Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Agent orchestration frameworks have proliferated, collectively exceeding 290,000 GitHub stars across LangGraph, CrewAI, Google ADK, OpenAI Agents SDK, Semantic Kernel, Strands, and LlamaIndex. All follow the same pattern: an external orchestrator above the LLM, injecting instructions and routing decisions every turn. Recent work has shown this architecture is dominated for procedural tasks by simply providing the procedure in a frontier model's system prompt [Dennis et al., 2026a], at the cost of consuming the context window, requiring a frontier model for every conversation, and exposing proprietary procedures to third-party providers. Compiling the procedure into the weights of a small fine-tuned model -- creating a subterranean agent -- should resolve all of these concerns, and prior work (SimpleTOD, FireAct, SynTOD, WorkflowLLM, Agent Lumos) has shown the technique works. Yet developer adoption has overwhelmingly favored orchestration. We identify three perceived barriers and address each empirically across travel booking (14 nodes), Zoom support (14 nodes, product-specific knowledge), and insurance claims (55 nodes, 6 decision hubs).
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 15 of its 22 references that are in the map. First posted on arXiv 21 May 2026.
Builds on
15 of 22 references.
Closest ancestors · 4
Same direction, newest first
- 2026ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditions3 Jan
- 2025Where LLM Agents Fail and How They can Learn From Failures29 Sep
- 2025Why Do Multi-Agent LLM Systems Fail?17 Mar
- 2024WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models8 Nov
Other references · 11
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
- 2020A Simple Language Model for Task-Oriented Dialoguecited by 331
- 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
- 2023FireAct: Toward Language Agent Fine-tuningcited by 247
- 2023Agent Lumos: Unified and Modular Training for Open-Source Language Agentscited by 72
- 2024LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Acceleratorscited by 31
- 2025Procedural Knowledge Improves Agentic LLM Workflowscited by 7
- 2024Simulating Task-Oriented Dialogues with State Transition Graphs and Large Language Modelscited by 6
- 2026In-Context Prompting Obsoletes Agent Orchestration for Procedural Taskscited by 2
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Failscited by 3
- 2026Procedural Knowledge Is Not Low-Rank: Why LoRA Fails to Internalize Multi-Step Procedurescited by 0
- 2026EvoIn: Bridging Evolution and Internalization for Agent Fine-Tuningcited by 0
- 2026Harness Annealing: Learning to Act with Less External Controlcited by 0
- 2026Harness Annealing: Learning to Act with Less External Control1 Oct 2026
- 2026EvoIn: Bridging Evolution and Internalization for Agent Fine-Tuning28 Sep 2026
- 2026Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails8 Sep 2026
- 2026Procedural Knowledge Is Not Low-Rank: Why LoRA Fails to Internalize Multi-Step Procedures23 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025A Declarative Language for Building And Orchestrating LLM-Powered Agent Workflows
- 2026Orchestral AI: A Framework for Agent Orchestration
- 2026Utility-Guided Agent Orchestration for Efficient LLM Tool Use
- 2026PerspectiveGap: A Benchmark for Multi-Agent Orchestration Prompting
- 2026Do More Agents Help? Controlled and Protocol-Aligned Evaluation of LLM Agent Workflows
- 2026Where Agent Frameworks Fall Short: Examining Functional Challenges and Usability Concerns
- 2026Pythia: Exploiting Workflow Predictability for Efficient Agent-Native LLM Serving
- 2026CURATE: Leveraging LLM Agents to Compose, Catalog, and Deploy Reproducible Workflows