# AIDev: Studying AI Coding Agents on GitHub

arXiv: https://arxiv.org/abs/2602.09185 · HTML: https://mlsanity.com/paper/2602.09185/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 27 of its 57 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 27 of 27)

- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [AutoCodeRover: Autonomous Program Improvement](https://mlsanity.com/paper/2404.05427/) (2024)
- [RepairAgent: An Autonomous, LLM-Based Agent for Program Repair](https://mlsanity.com/paper/2403.17134/) (2024)
- [Grounded Copilot: How Programmers Interact with Code-Generating Models](https://mlsanity.com/paper/2206.15000/) (2022)
- [An Analysis of the Automatic Bug Fixing Performance of ChatGPT](https://mlsanity.com/paper/2301.08653/) (2023)
- [Productivity Assessment of Neural Code Completion](https://mlsanity.com/paper/2205.06537/) (2022)
- [HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scale](https://mlsanity.com/paper/2409.16299/) (2024)
- [Alibaba LingmaAgent: Improving Automated Issue Resolution via Comprehensive Repository Exploration](https://mlsanity.com/paper/2406.01422/) (2024)
- [Agentic Software Engineering: Foundational Pillars and a Research Roadmap](https://mlsanity.com/paper/2509.06216/) (2025)
- [Assistance or Disruption? Exploring and Evaluating the Design and Trade-offs of Proactive AI Programming Support](https://mlsanity.com/paper/2502.18658/) (2025)
- [The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers](https://mlsanity.com/paper/2404.02806/) (2024)
- [Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap](https://mlsanity.com/paper/2410.06107/) (2024)
- [Beyond Code Generation: An Observational Study of ChatGPT Usage in Software Engineering Practice](https://mlsanity.com/paper/2404.14901/) (2024)
- [Unlocking Reasoning Potential in Large Langauge Models by Scaling Code-form Planning](https://mlsanity.com/paper/2409.12452/) (2024)
- [Automating Dependency Updates in Practice: An Exploratory Study on GitHub Dependabot](https://mlsanity.com/paper/2206.07230/) (2022)
- [The Impact of Large Language Models on Open-source Innovation: Evidence from GitHub Copilot](https://mlsanity.com/paper/2409.08379/) (2024)
- [A Study on Developer Behaviors for Validating and Repairing LLM-Generated Code Using Eye Tracking and IDE Actions](https://mlsanity.com/paper/2405.16081/) (2024)
- [How to Design a Program Repair Bot? Insights from the Repairnator Project](https://mlsanity.com/paper/1811.09852/) (2018)
- [An Empirical Study of Bots in Software Development -- Characteristics and Challenges from a Practitioner's Perspective](https://mlsanity.com/paper/2005.13969/) (2020)
- [Don't Disturb Me: Challenges of Interacting with SoftwareBots on Open Source Software Projects](https://mlsanity.com/paper/2103.13950/) (2021)
- [When Developer Aid Becomes Security Debt: A Systematic Analysis of Insecure Behaviors in LLM Coding Agents](https://mlsanity.com/paper/2507.09329/) (2025)
- [Bots Don't Mind Waiting, Do They? Comparing the Interaction With Automatically and Manually Created Pull Requests](https://mlsanity.com/paper/2103.03591/) (2021)
- [NoCodeGPT: A No-Code Interface for Building Web Apps with Language Models](https://mlsanity.com/paper/2310.14843/) (2023)
- [Exploring the Challenges and Opportunities of AI-assisted Codebase Generation](https://mlsanity.com/paper/2508.07966/) (2025)
- [Emotional Strain and Frustration in LLM Interactions in Software Engineering](https://mlsanity.com/paper/2504.10050/) (2025)
- [The Present and Future of Bots in Software Engineering](https://mlsanity.com/paper/2207.01254/) (2022)
- [Beyond Code: The Multidimensional Impacts of Large Language Models in Software Development](https://mlsanity.com/paper/2506.22704/) (2025)

## Built on by (most cited first, 17 of 17)

- [Collaborator or Assistant? How AI Coding Agents Partition Work Across Pull Request Lifecycles](https://mlsanity.com/paper/2605.08017/) (2026)
- [Agentic Agile-V: From Vibe Coding to Verified Engineering in Software and Hardware Development](https://mlsanity.com/paper/2605.20456/) (2026)
- [All Smoke, No Alarm: Oracle Signals in Agent-Authored Test Code](https://mlsanity.com/paper/2606.18168/) (2026)
- [AI Agent Pull Requests on GitHub: Frequency, Structure, and Merge Conflict Rates](https://mlsanity.com/paper/2607.04697/) (2026)
- [A First Look at Coding Agents' Compliance with AI Contribution Rules in Open-Source Communities](https://mlsanity.com/paper/2607.26819/) (2026)
- [An Exploratory Study of Agent Plans for Agentic AI Coding Tools in Open-Source Software](https://mlsanity.com/paper/2608.04661/) (2026)
- [Where Accountability Lives: Mapping Human Responsibility to Workflow Artifacts in Agentic Software Development](https://mlsanity.com/paper/2608.15678/) (2026)
- [From Assistance to Agency: Rethinking Autonomy and Control in CI/CD Pipelines](https://mlsanity.com/paper/2605.07062/) (2026)
- [Quality and Security Signals in AI-Generated Python Refactoring Pull Requests](https://mlsanity.com/paper/2605.21453/) (2026)
- [Software Delegation Contracts: Measuring Reviewability in AI Coding-Agent Work](https://mlsanity.com/paper/2606.17099/) (2026)
- [Detecting AI Coding Agents in Open Source: A Validated Multi-Method Census of 180 Million Repositories](https://mlsanity.com/paper/2606.24429/) (2026)
- [Govern the Repository, Not the Agent: Measuring Ecosystem-Level Risk in AI-Native Software](https://mlsanity.com/paper/2606.28235/) (2026)
- [From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation](https://mlsanity.com/paper/2608.20195/) (2026)
- [On the Maintenance and Co-evolution of Agent Plugins: An Empirical Study of Claude Code Plugin Marketplaces](https://mlsanity.com/paper/2608.28497/) (2026)
- [Who Finishes the Job? A Study of Follow-Up Fixes and Commit Authorship on AI Coding Agent Pull Requests](https://mlsanity.com/paper/2609.26847/) (2026)
- [Between the Commits: Process, Error, and Claim Reliability in a Wholly AI-Authored Codebase](https://mlsanity.com/paper/2609.29744/) (2026)
- [Co-Modifying Application Code and Deployment Infrastructure: A Causal Study of AI Agent Pull Requests](https://mlsanity.com/paper/2610.12849/) (2026)

## Abstract (arXiv)

AI coding agents are rapidly transforming software engineering by performing tasks such as feature development, debugging, and testing. Despite their growing impact, the research community lacks a comprehensive dataset capturing how these agents are used in real-world projects. To address this gap, we introduce AIDev, a large-scale dataset focused on agent-authored pull requests (Agentic-PRs) in real-world GitHub repositories. AIDev aggregates 932,791 Agentic-PRs produced by five agents: OpenAI Codex, Devin, GitHub Copilot, Cursor, and Claude Code. These PRs span 116,211 repositories and involve 72,189 developers. In addition, AIDev includes a curated subset of 33,596 Agentic-PRs from 2,807 repositories with over 100 stars, providing further information such as comments, reviews, commits, and related issues. This dataset offers a foundation for future research on AI adoption, developer productivity, and human-AI collaboration in the new era of software engineering. > AI Agent, Agentic AI, Coding Agent, Agentic Coding, Agentic Software Engineering, Agentic Engineering