# Compiled AI: Deterministic Code Generation for LLM-Based Workflow Automation

arXiv: https://arxiv.org/abs/2604.05150 · HTML: https://mlsanity.com/paper/2604.05150/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 41 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Builds on (most cited first, 12 of 12)

- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [Efficient Guided Generation for Large Language Models](https://mlsanity.com/paper/2307.09702/) (2023)
- [An Empirical Study of the Non-determinism of ChatGPT in Code Generation](https://mlsanity.com/paper/2308.02828/) (2023)
- [Non-Determinism of "Deterministic" LLM Settings](https://mlsanity.com/paper/2408.04667/) (2024)
- [DocILE Benchmark for Document Information Localization and Extraction](https://mlsanity.com/paper/2302.05658/) (2023)
- [FlowMind: Automatic Workflow Generation with LLMs](https://mlsanity.com/paper/2404.13050/) (2024)
- [Blueprint First, Model Second: A Framework for Deterministic LLM Workflow](https://mlsanity.com/paper/2508.02721/) (2025)
- [Towards a HIPAA Compliant Agentic AI System in Healthcare](https://mlsanity.com/paper/2504.17669/) (2025)
- [Beyond Synthetic Benchmarks: Evaluating LLM Performance on Real-World Class-Level Code Generation](https://mlsanity.com/paper/2510.26130/) (2025)

## Built on by (most cited first, 4 of 4)

- [PreAct: Computer-Using Agents that Get Faster on Repeated Tasks](https://mlsanity.com/paper/2606.17929/) (2026)
- [SPL: Orchestrating Workflows with Declarative Deterministic-Probabilistic Composition](https://mlsanity.com/paper/2607.07727/) (2026)
- [Generating Workflow DAGs from Natural Language with Non-Reasoning LLMs](https://mlsanity.com/paper/2608.30250/) (2026)
- [When to Compile a Computer-Use Agent? Measuring Payback and Making Compilation Decisions for Token Efficiency](https://mlsanity.com/paper/2610.02932/) (2026)

## Abstract (arXiv)

We study compiled AI, a paradigm in which large language models generate executable code artifacts during a compilation phase, after which workflows execute deterministically without further model invocation. This paradigm has antecedents in prior work on declarative pipeline optimization (DSPy) and hybrid neural-symbolic planning (LLM+P); our contribution is a systems-oriented study of its application to high-stakes enterprise workflows, with particular emphasis on healthcare settings where reliability and auditability are critical. By constraining generation to narrow business-logic functions embedded in validated templates, compiled AI trades runtime flexibility for predictability, auditability, cost efficiency, and reduced security exposure. We introduce (i) a system architecture for constrained LLM-based code generation, (ii) a four-stage generation-and-validation pipeline that converts probabilistic model output into production-ready code artifacts, and (iii) an evaluation framework measuring operational metrics including token amortization, determinism, reliability, security, and cost. We evaluate on two task types: function-calling (BFCL, n=400) and document intelligence (DocILE, n=5,680 invoices). On function-calling, compiled AI achieves 96% task completion with zero execution tokens, breaking even with runtime inference at approximately 17 transactions and reducing token consumption by 57x at 1,000 transactions. On document intelligence, our Code Factory variant matches Direct LLM on key field extraction (KILE: 80.0%) while achieving the highest line item recognition accuracy (LIR: 80.4%). Security evaluation across 135 test cases demonstrates 96.7% accuracy on prompt injection detection and 87.5% on static code safety analysis with zero false positives.