# How AI Coding Agents Modify Code: A Large-Scale Study of GitHub Pull Requests

arXiv: https://arxiv.org/abs/2601.17581 · HTML: https://mlsanity.com/paper/2601.17581/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 13 of its 44 references that are in the map.
- First posted on arXiv 24 Jan 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 13 of 13)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [GraphCodeBERT: Pre-training Code Representations with Data Flow](https://mlsanity.com/paper/2009.08366/) (2020)
- [Bugs in Large Language Models Generated Code: An Empirical Study](https://mlsanity.com/paper/2403.08937/) (2024)
- [On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub](https://mlsanity.com/paper/2509.14745/) (2025)
- [Agentic Software Engineering: Foundational Pillars and a Research Roadmap](https://mlsanity.com/paper/2509.06216/) (2025)
- [Teach AI How to Code: Using Large Language Models as Teachable Agents for Programming Education](https://mlsanity.com/paper/2309.14534/) (2023)
- [Towards AI-Native Software Engineering (SE 3.0): A Vision and a Challenge Roadmap](https://mlsanity.com/paper/2410.06107/) (2024)
- [Evaluating Software Development Agents: Patch Patterns, Code Quality, and Issue Complexity in Real-World GitHub Scenarios](https://mlsanity.com/paper/2410.12468/) (2024)
- [An Empirical Study of Testing Practices in Open Source AI Agent Frameworks and Agentic Applications](https://mlsanity.com/paper/2509.19185/) (2025)
- [Improved IR-based Bug Localization with Intelligent Relevance Feedback](https://mlsanity.com/paper/2501.10542/) (2025)
- [An Empirical Study on Commit Message Generation using LLMs via In-Context Learning](https://mlsanity.com/paper/2502.18904/) (2025)
- [Refactoring-Aware Patch Integration Across Structurally Divergent Java Forks](https://mlsanity.com/paper/2508.06718/) (2025)

## Built on by (most cited first, 8 of 8)

- [AgenticFlict: A Large-Scale Dataset of Merge Conflicts in AI Coding Agent Pull Requests on GitHub](https://mlsanity.com/paper/2604.03551/) (2026)
- [AI Agent Pull Requests on GitHub: Frequency, Structure, and Merge Conflict Rates](https://mlsanity.com/paper/2607.04697/) (2026)
- [AIRA: AI-Induced Risk Audit: A Structured Inspection Framework for AI-Generated Code](https://mlsanity.com/paper/2604.17587/) (2026)
- [Augmentation with Dilution: A Large-Scale Empirical Study of Human Contributor Ecosystems After AI Coding Agent Adoption](https://mlsanity.com/paper/2606.26289/) (2026)
- [Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents](https://mlsanity.com/paper/2607.12428/) (2026)
- [How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests](https://mlsanity.com/paper/2607.21832/) (2026)
- [AgenTag: Attribution of AI Coding Agents from Behavioral Fingerprints](https://mlsanity.com/paper/2608.00966/) (2026)
- [Who Finishes the Job? A Study of Follow-Up Fixes and Commit Authorship on AI Coding Agent Pull Requests](https://mlsanity.com/paper/2609.26847/) (2026)

## Abstract (arXiv)

AI coding agents are increasingly acting as autonomous contributors by generating and submitting pull requests (PRs). However, we lack empirical evidence on how these agent-generated PRs differ from human contributions, particularly in how they modify code and describe their changes. Understanding these differences is essential for assessing their reliability and impact on development workflows. Using the MSR 2026 Mining Challenge version of the AIDev dataset, we analyze 24,014 merged Agentic PRs (440,295 commits) and 5,081 merged Human PRs (23,242 commits). We examine additions, deletions, commits, and files touched, and evaluate the consistency between PR descriptions and their diffs using lexical and semantic similarity. Agentic PRs differ substantially from Human PRs in commit count (Cliff's δ = 0.5429) and show moderate differences in files touched and deleted lines. They also exhibit slightly higher description-to-diff similarity across all measures. These findings provide a large-scale empirical characterization of how AI coding agents contribute to open source development.