# AutoDev: Automated AI-Driven Development

arXiv: https://arxiv.org/abs/2403.08299 · HTML: https://mlsanity.com/paper/2403.08299/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 16 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://mlsanity.com/paper/2201.11990/) (2022)
- [CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion](https://mlsanity.com/paper/2310.11248/) (2023)

## Built on by (most cited first, 24 of 24)

- [Agent-as-a-Judge: Evaluate Agents with Agents](https://mlsanity.com/paper/2410.10934/) (2024)
- [MASAI: Modular Architecture for Software-engineering AI Agents](https://mlsanity.com/paper/2406.11638/) (2024)
- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)
- [Evaluating Agent-based Program Repair at Google](https://mlsanity.com/paper/2501.07531/) (2025)
- [RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code](https://mlsanity.com/paper/2503.07832/) (2025)
- [Agent READMEs: An Empirical Study of Context Files for Agentic Coding](https://mlsanity.com/paper/2511.12884/) (2025)
- [Unified Software Engineering Agent as AI Software Engineer](https://mlsanity.com/paper/2506.14683/) (2025)
- [Causal Evaluation of Language Models](https://mlsanity.com/paper/2405.00622/) (2024)
- [From Code Foundation Models to Agents and Applications: A Comprehensive Survey and Practical Guide to Code Intelligence](https://mlsanity.com/paper/2511.18538/) (2025)
- [On the Use of Agentic Coding Manifests: An Empirical Study of Claude Code](https://mlsanity.com/paper/2509.14744/) (2025)
- [Collaborative Evolving Strategy for Automatic Data-Centric Development](https://mlsanity.com/paper/2407.18690/) (2024)
- [TableTalk: Scaffolding Spreadsheet Development with a Language Agent](https://mlsanity.com/paper/2502.09787/) (2025)
- [OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Models](https://mlsanity.com/paper/2505.21537/) (2025)
- [Designing LLM-based Multi-Agent Systems for Software Engineering Tasks: Quality Attributes, Design Patterns and Rationale](https://mlsanity.com/paper/2511.08475/) (2025)
- [Why Human Guidance Matters in Collaborative Vibe Coding](https://mlsanity.com/paper/2602.10473/) (2026)
- [How AI Coding Agents Communicate: A Study of Pull Request Description Characteristics and Human Review Responses](https://mlsanity.com/paper/2602.17084/) (2026)
- [Empowering Real-World: A Survey on the Technology, Practice, and Evaluation of LLM-driven Industry Agents](https://mlsanity.com/paper/2510.17491/) (2025)
- [Who Broke the System? Failure Localization in LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2607.07989/) (2026)
- [Morescient GAI for Software Engineering (Extended Version)](https://mlsanity.com/paper/2406.04710/) (2024)
- [Multi-Agent Game Generation and Evaluation via Audio-Visual Recordings](https://mlsanity.com/paper/2508.00632/) (2025)
- [Debug2Fix: Can Interactive Debugging Help Coding Agents Fix More Bugs?](https://mlsanity.com/paper/2602.18571/) (2026)
- [Rethinking Code Review in the Age of AI: A Vision for Agentic Code Review](https://mlsanity.com/paper/2605.17548/) (2026)
- [Governance Controls for AI-Generated Test Artifacts in Autonomous Software Testing](https://mlsanity.com/paper/2606.08806/) (2026)
- [From Overload to Insights: How AI Agents Can Support Scientists in Analyzing Complex Data](https://mlsanity.com/paper/2607.16845/) (2026)

## Abstract (arXiv)

The landscape of software development has witnessed a paradigm shift with the advent of AI-powered assistants, exemplified by GitHub Copilot. However, existing solutions are not leveraging all the potential capabilities available in an IDE such as building, testing, executing code, git operations, etc. Therefore, they are constrained by their limited capabilities, primarily focusing on suggesting code snippets and file manipulation within a chat-based interface. To fill this gap, we present AutoDev, a fully automated AI-driven software development framework, designed for autonomous planning and execution of intricate software engineering tasks. AutoDev enables users to define complex software engineering objectives, which are assigned to AutoDev's autonomous AI Agents to achieve. These AI agents can perform diverse operations on a codebase, including file editing, retrieval, build processes, execution, testing, and git operations. They also have access to files, compiler output, build and testing logs, static analysis tools, and more. This enables the AI Agents to execute tasks in a fully automated manner with a comprehensive understanding of the contextual information required. Furthermore, AutoDev establishes a secure development environment by confining all operations within Docker containers. This framework incorporates guardrails to ensure user privacy and file security, allowing users to define specific permitted or restricted commands and operations within AutoDev. In our evaluation, we tested AutoDev on the HumanEval dataset, obtaining promising results with 91.5% and 87.8% of Pass@1 for code generation and test generation respectively, demonstrating its effectiveness in automating software engineering tasks while maintaining a secure and user-controlled development environment.