StackPilot: Autonomous Function Agents for Scalable and Environment-Free Code Execution
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Recent advances in large language models (LLMs) have substantially enhanced automated code generation across a wide range of programming languages. Nonetheless, verifying the correctness and executability of LLM-generated code remains a significant challenge, as traditional methods rely on language-specific compilers and environment-dependent runtimes. To overcome these limitations, we introduce StackPilot, an LLM-native, multi-agent framework designed for language-agnostic code verification and execution, which operates independently of conventional toolchains. StackPilot offers three principal innovations: (1) a Function-as-Agents paradigm, in which each function is modeled as an autonomous agent capable of fine-grained reasoning and collaborative verification; (2) an LLM-as-Executor strategy, which enables scalable verification via stack-based scheduling; and (3) a novel snapshot mechanism that preserves complete execution contexts, facilitating deterministic and lossless context switching during verification. Empirical evaluations demonstrate that StackPilot achieves framework reliability rates between 89% and 97%, substantially outperforming baseline approaches. These results indicate that StackPilot can reliably verify and execute a significantly larger proportion of LLM-generated code across diverse programming tasks compared to existing methods.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 35 of its 52 references that are in the map. First posted on arXiv 6 Aug 2025.
Builds on
35 of 52 references.
Closest ancestors · 3
Same direction, newest first
- 2024AIOS Compiler: LLM as Interpreter for Natural Language Programming and Flow Programming of AI Agents11 May
- 2023ChatDev: Communicative Agents for Software Development16 Jul
- 2023Self-collaboration Code Generation via ChatGPT15 Apr
Other references · 32
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,761
- 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
- 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,332
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
- 2021Measuring Coding Challenge Competence With APPScited by 879
- 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
20 more
- 2023StarCoder: may the source be with you!cited by 790
- 2024A Survey on Large Language Models for Code Generationcited by 646
- 2023AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviorscited by 580
- 2022InCoder: A Generative Model for Code Infilling and Synthesiscited by 492
- 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 325
- 2022GitHub Copilot AI pair programmer: Asset or Liability?cited by 150
- 2023Scalable Multi-Robot Collaboration with Large Language Models: Centralized or Decentralized Systems?cited by 104
- 2023Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issuescited by 78
- 2023Can ChatGPT replace StackOverflow? A Study on Robustness and Reliability of Large Language Model Code Generationcited by 72
- 2023Lyfe Agents: Generative agents for low-cost real-time social interactionscited by 49
- 2024A Survey on Evaluating Large Language Models in Code Generation Taskscited by 43
- 2023LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystemcited by 35
- 2024Beyond Language Models: Byte Models are Digital World Simulatorscited by 21
- 2022FixEval: Execution-based Evaluation of Program Fixes for Programming Problemscited by 17
- 2025MASTER: A Multi-Agent System with LLM Specialized MCTScited by 15
- 2024From Commands to Prompts: LLM-based Semantic File System for AIOScited by 14
- 2024Large Language Models as Code Executors: An Exploratory Studycited by 12
- 2025Cerebrum (AIOS SDK): A Platform for Agent Development, Deployment, Distribution, and Discoverycited by 6
- 2025A Preliminary Study on the Robustness of Code Generation by Large Language Modelscited by 5
- 2025Planet as a Brain: Towards Internet of AgentSites based on AIOS Servercited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025AdaCoder: An Adaptive Planning and Multi-Agent Framework for Function-Level Code Generation
- 2025TypePilot: Leveraging the Scala Type System for Secure LLM-generated Code
- 2025CodeARC: Benchmarking Reasoning Capabilities of LLM Agents for Inductive Program Synthesis
- 2026AgentForge: Execution-Grounded Multi-Agent LLM Framework for Autonomous Software Engineering
- 2024Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization
- 2026ExecuGraph: A Multi-Agent, Execution-Grounded Framework for Reliable Backend Code Synthesis with Large Language Models
- 2026WybeCoder: Verified Imperative Code Generation
- 2025Enhancing LLM Code Generation: A Systematic Evaluation of Multi-Agent Collaboration and Runtime Debugging for Improved Accuracy, Reliability, and Latency