# AIOS Compiler: LLM as Interpreter for Natural Language Programming and Flow Programming of AI Agents arXiv: https://arxiv.org/abs/2405.06907 ยท HTML: https://mlsanity.com/paper/2405.06907/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. - Builds on 33 of its 51 references that are in the map. - First posted on arXiv 11 May 2024. ## Placed in - LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/) ## Builds on (most cited first, 30 of 33) - [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022) - [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020) - [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022) - [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022) - [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019) - [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022) - [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023) - [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023) - [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021) - [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022) - [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023) - [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023) - [Improving language models by retrieving from trillions of tokens](https://mlsanity.com/paper/2112.04426/) (2021) - [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022) - [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022) - [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023) - [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022) - [Reasoning with Language Model is Planning with World Model](https://mlsanity.com/paper/2305.14992/) (2023) - [DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines](https://mlsanity.com/paper/2310.03714/) (2023) - [LLM+P: Empowering Large Language Models with Optimal Planning Proficiency](https://mlsanity.com/paper/2304.11477/) (2023) - [Visual Transformers: Token-based Image Representation and Processing for Computer Vision](https://mlsanity.com/paper/2006.03677/) (2020) - [Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP](https://mlsanity.com/paper/2212.14024/) (2022) - [REFINER: Reasoning Feedback on Intermediate Representations](https://mlsanity.com/paper/2304.01904/) (2023) - [OpenAGI: When LLM Meets Domain Experts](https://mlsanity.com/paper/2304.04370/) (2023) - [TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs](https://mlsanity.com/paper/2303.16434/) (2023) - [Task and Motion Planning with Large Language Models for Object Rearrangement](https://mlsanity.com/paper/2303.06247/) (2023) - [The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Development](https://mlsanity.com/paper/2302.07080/) (2023) - [Program Synthesis using Natural Language](https://mlsanity.com/paper/1509.00413/) (2015) - [Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agents](https://mlsanity.com/paper/2402.00798/) (2024) - [LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystem](https://mlsanity.com/paper/2312.03815/) (2023) ## Built on by (most cited first, 8 of 8) - [SEW: Self-Evolving Agentic Workflows for Automated Code Generation](https://mlsanity.com/paper/2505.18646/) (2025) - [SPIO: Ensemble and Selective Strategies via LLM-Based Multi-Agent Planning in Automated Data Science](https://mlsanity.com/paper/2503.23314/) (2025) - [EVALOOOP: A Self-Consistency-Centered Framework for Assessing Large Language Model Robustness in Programming](https://mlsanity.com/paper/2505.12185/) (2025) - [AutoGRAMS: Autonomous Graphical Agent Modeling Software](https://mlsanity.com/paper/2407.10049/) (2024) - [StackPilot: Autonomous Function Agents for Scalable and Environment-Free Code Execution](https://mlsanity.com/paper/2508.11665/) (2025) - [PADME: Procedure Aware DynaMic Execution](https://mlsanity.com/paper/2510.11281/) (2025) - [The New Compiler Stack: A Survey on the Synergy of LLMs and Compilers](https://mlsanity.com/paper/2601.02045/) (2026) - [Beyond Human-Readable: Rethinking Software Engineering Conventions for the Agentic Development Era](https://mlsanity.com/paper/2604.07502/) (2026) ## Abstract (arXiv) Since their inception, programming languages have trended towards greater readability and lower barriers for programmers. Following this trend, natural language can be a promising type of programming language that provides great flexibility and usability and helps towards the democracy of programming. However, the inherent vagueness, ambiguity, and verbosity of natural language pose significant challenges in developing an interpreter that can accurately understand the programming logic and execute instructions written in natural language. Fortunately, recent advancements in Large Language Models (LLMs) have demonstrated remarkable proficiency in interpreting complex natural language. Inspired by this, we develop a novel system for Code Representation and Execution (CoRE), which employs LLM as interpreter to interpret and execute natural language instructions. The proposed system unifies natural language programming, pseudo-code programming, and flow programming under the same representation for constructing language agents, while LLM serves as the interpreter to interpret and execute the agent programs. In this paper, we begin with defining the programming syntax that structures natural language instructions logically. During the execution, we incorporate external memory to minimize redundancy. Furthermore, we equip the designed interpreter with the capability to invoke external tools, compensating for the limitations of LLM in specialized domains or when accessing real-time information. This work is open-source at https://github.com/agiresearch/CoRE, https://github.com/agiresearch/OpenAGI, and https://github.com/agiresearch/AIOS.