# The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering

arXiv: https://arxiv.org/abs/2507.15003 · HTML: https://mlsanity.com/paper/2507.15003/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 36 references that are in the map.
- First posted on arXiv 20 Jul 2025.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 17 of 17)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?](https://mlsanity.com/paper/2410.03859/) (2024)
- [The SWE-Bench Illusion: When State-of-the-Art LLMs Remember Instead of Reason](https://mlsanity.com/paper/2506.12286/) (2025)
- [LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarks](https://mlsanity.com/paper/2502.06215/) (2025)
- [Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap](https://mlsanity.com/paper/2410.06107/) (2024)
- [Agentic Bug Reproduction for Effective Automated Program Repair at Google](https://mlsanity.com/paper/2502.01821/) (2025)
- [Agentic AI Software Engineers: Programming with Trust](https://mlsanity.com/paper/2502.13767/) (2025)
- [Unified Software Engineering Agent as AI Software Engineer](https://mlsanity.com/paper/2506.14683/) (2025)
- [SPICE: An Automated SWE-Bench Labeling Pipeline for Issue Clarity, Test Coverage, and Effort Estimation](https://mlsanity.com/paper/2507.09108/) (2025)
- [FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Tasks](https://mlsanity.com/paper/2504.06939/) (2025)
- [Can Language Models Replace Programmers for Coding? REPOCOD Says 'Not Yet'](https://mlsanity.com/paper/2410.21647/) (2024)
- [Understanding User Mental Models in AI-Driven Code Completion Tools: Insights from an Elicitation Study](https://mlsanity.com/paper/2502.02194/) (2025)
- [Chain of Draft for Software Engineering: Challenges in Applying Concise Reasoning to Code Tasks](https://mlsanity.com/paper/2506.10987/) (2025)
- [Automated Code Review In Practice](https://mlsanity.com/paper/2412.18531/) (2024)
- [Towards an Interpretable Analysis for Estimating the Resolution Time of Software Issues](https://mlsanity.com/paper/2505.01108/) (2025)

## Built on by (most cited first, 9 of 9)

- [VisCoder2: Building Multi-Language Visualization Coding Agents](https://mlsanity.com/paper/2510.23642/) (2025)
- [Harness Engineering: Anatomy, Architecture, and Evolution of Coding Agents -- A Source-Code Study of Eleven Systems](https://mlsanity.com/paper/2609.00006/) (2026)
- [HAFixAgent: History-Aware Program Repair Agent](https://mlsanity.com/paper/2511.01047/) (2025)
- [Beyond Functional Correctness: Design Issues in AI IDE-Generated Large-Scale Projects](https://mlsanity.com/paper/2604.06373/) (2026)
- [Rule Taxonomy and Evolution in AI IDEs: A Mining and Survey Study](https://mlsanity.com/paper/2606.12231/) (2026)
- [Do programming languages still matter to your AI coding agent teammate? Evidence at scale from chess engines](https://mlsanity.com/paper/2606.13763/) (2026)
- [AI Agent Pull Requests on GitHub: Frequency, Structure, and Merge Conflict Rates](https://mlsanity.com/paper/2607.04697/) (2026)
- [3100 Opinions on Code Review in an AI World: Building Causal Theory from Practitioner Discourse](https://mlsanity.com/paper/2607.07980/) (2026)
- [LLM Serving in the Wild: An Empirical Study of Frameworks, Methods, and System Designs](https://mlsanity.com/paper/2608.03036/) (2026)

## Abstract (arXiv)

The future of software engineering--SE 3.0--is unfolding with the rise of AI teammates: autonomous, goal-driven systems collaborating with human developers. Among these, autonomous coding agents are especially transformative, now actively initiating, reviewing, and evolving code at scale. This paper introduces AIDev, the first large-scale dataset capturing how such agents operate in the wild. Spanning over 456,000 pull requests by five leading agents--OpenAI Codex, Devin, GitHub Copilot, Cursor, and Claude Code--across 61,000 repositories and 47,000 developers, AIDev provides an unprecedented empirical foundation for studying autonomous teammates in software development. Unlike prior work that has largely theorized the rise of AI-native software engineering, AIDev offers structured, open data to support research in benchmarking, agent readiness, optimization, collaboration modeling, and AI governance. The dataset includes rich metadata on PRs, authorship, review timelines, code changes, and integration outcomes--enabling exploration beyond synthetic benchmarks like SWE-bench. For instance, although agents often outperform humans in speed, their PRs are accepted less frequently, revealing a trust and utility gap. Furthermore, while agents accelerate code submission--one developer submitted as many PRs in three days as they had in three years--these are structurally simpler (via code complexity metrics). We envision AIDev as a living resource: extensible, analyzable, and ready for the SE and AI communities. Grounding SE 3.0 in real-world evidence, AIDev enables a new generation of research into AI-native workflows and supports building the next wave of symbiotic human-AI collaboration. The dataset is publicly available at https://github.com/SAILResearch/AI_Teammates_in_SE3. > AI Agent, Agentic AI, Coding Agent, Agentic Coding, Software Engineering Agent