# Compiling Agentic Workflows into LLM Weights: Near-Frontier Quality at Two Orders of Magnitude Less Cost

arXiv: https://arxiv.org/abs/2605.22502 · HTML: https://mlsanity.com/paper/2605.22502/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 15 of its 22 references that are in the map.
- First posted on arXiv 21 May 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 15 of 15)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [AgentTuning: Enabling Generalized Agent Abilities for LLMs](https://mlsanity.com/paper/2310.12823/) (2023)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [FireAct: Toward Language Agent Fine-tuning](https://mlsanity.com/paper/2310.05915/) (2023)
- [Where LLM Agents Fail and How They can Learn From Failures](https://mlsanity.com/paper/2509.25370/) (2025)
- [Agent Lumos: Unified and Modular Training for Open-Source Language Agents](https://mlsanity.com/paper/2311.05657/) (2023)
- [ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditions](https://mlsanity.com/paper/2601.06112/) (2026)
- [LLM-Inference-Bench: Inference Benchmarking of Large Language Models on AI Accelerators](https://mlsanity.com/paper/2411.00136/) (2024)
- [WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models](https://mlsanity.com/paper/2411.05451/) (2024)
- [Procedural Knowledge Improves Agentic LLM Workflows](https://mlsanity.com/paper/2511.07568/) (2025)
- [Simulating Task-Oriented Dialogues with State Transition Graphs and Large Language Models](https://mlsanity.com/paper/2404.14772/) (2024)
- [In-Context Prompting Obsoletes Agent Orchestration for Procedural Tasks](https://mlsanity.com/paper/2604.27891/) (2026)

## Built on by (most cited first, 5 of 5)

- [Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails](https://mlsanity.com/paper/2609.09134/) (2026)
- [Procedural Knowledge Is Not Low-Rank: Why LoRA Fails to Internalize Multi-Step Procedures](https://mlsanity.com/paper/2607.21612/) (2026)
- [EvoIn: Bridging Evolution and Internalization for Agent Fine-Tuning](https://mlsanity.com/paper/2609.35290/) (2026)
- [Harness Annealing: Learning to Act with Less External Control](https://mlsanity.com/paper/2610.01235/) (2026)
- [Learning What to Investigate Next: Meta-Reasoning for Long-Horizon Research Agents](https://mlsanity.com/paper/2610.02525/) (2026)

## Abstract (arXiv)

Agent orchestration frameworks have proliferated, collectively exceeding 290,000 GitHub stars across LangGraph, CrewAI, Google ADK, OpenAI Agents SDK, Semantic Kernel, Strands, and LlamaIndex. All follow the same pattern: an external orchestrator above the LLM, injecting instructions and routing decisions every turn. Recent work has shown this architecture is dominated for procedural tasks by simply providing the procedure in a frontier model's system prompt [Dennis et al., 2026a], at the cost of consuming the context window, requiring a frontier model for every conversation, and exposing proprietary procedures to third-party providers. Compiling the procedure into the weights of a small fine-tuned model -- creating a subterranean agent -- should resolve all of these concerns, and prior work (SimpleTOD, FireAct, SynTOD, WorkflowLLM, Agent Lumos) has shown the technique works. Yet developer adoption has overwhelmingly favored orchestration. We identify three perceived barriers and address each empirically across travel booking (14 nodes), Zoom support (14 nodes, product-specific knowledge), and insurance claims (55 nodes, 6 decision hubs).