# Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHub

arXiv: https://arxiv.org/abs/2601.00477 · HTML: https://mlsanity.com/paper/2601.00477/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 32 of its 60 references that are in the map.
- First posted on arXiv 1 Jan 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models](https://mlsanity.com/paper/2312.04724/) (2023)
- [On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub](https://mlsanity.com/paper/2509.14745/) (2025)
- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)
- [Agentic Software Engineering: Foundational Pillars and a Research Roadmap](https://mlsanity.com/paper/2509.06216/) (2025)
- [Knowledge Boundary of Large Language Models: A Survey](https://mlsanity.com/paper/2412.12472/) (2024)
- [SALLM: Security Assessment of Generated Code](https://mlsanity.com/paper/2311.00889/) (2023)
- [AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities](https://mlsanity.com/paper/2508.11126/) (2025)
- [CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology](https://mlsanity.com/paper/2402.01411/) (2024)
- [Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Tasks](https://mlsanity.com/paper/2507.15821/) (2025)
- [An Empirical Study of Testing Practices in Open Source AI Agent Frameworks and Agentic Applications](https://mlsanity.com/paper/2509.19185/) (2025)
- [FRANC: A Lightweight Framework for High-Quality Code Generation](https://mlsanity.com/paper/2307.08220/) (2023)
- [Agentsway -- Software Development Methodology for AI Agents-based Teams](https://mlsanity.com/paper/2510.23664/) (2025)
- [Taming Uncertainty via Automation: Observing, Analyzing, and Optimizing Agentic AI Systems](https://mlsanity.com/paper/2507.11277/) (2025)
- [Does Code Quality Affect Pull Request Acceptance? An empirical study](https://mlsanity.com/paper/1908.09321/) (2019)
- [ProofWright: Towards Agentic Formal Verification of CUDA](https://mlsanity.com/paper/2511.12294/) (2025)
- [Facilitating Trustworthy Human-Agent Collaboration in LLM-based Multi-Agent System oriented Software Engineering](https://mlsanity.com/paper/2505.04251/) (2025)
- [Pull Request Latency Explained: An Empirical Overview](https://mlsanity.com/paper/2108.09946/) (2021)
- [Experimenting with Multi-Agent Software Development: Towards a Unified Platform](https://mlsanity.com/paper/2406.05381/) (2024)
- [ChatCollab: Exploring Collaboration Between Humans and AI Agents in Software Teams](https://mlsanity.com/paper/2412.01992/) (2024)
- [Formalizing the Safety, Security, and Functional Properties of Agentic AI Systems](https://mlsanity.com/paper/2510.14133/) (2025)
- [Toward Agentic Software Engineering Beyond Code: Framing Vision, Values, and Vocabulary](https://mlsanity.com/paper/2510.19692/) (2025)

## Built on by (most cited first, 7 of 7)

- [AI Agent Pull Requests on GitHub: Frequency, Structure, and Merge Conflict Rates](https://mlsanity.com/paper/2607.04697/) (2026)
- [Setup Complete, Now You Are Compromised: Weaponizing Setup Instructions Against AI Coding Agents](https://mlsanity.com/paper/2607.15143/) (2026)
- [Terminal Agents: A Survey of AI Agents in Command-Line Environments](https://mlsanity.com/paper/2608.20485/) (2026)
- [An Empirical Study of Security Calibration in Large Language Models for Code](https://mlsanity.com/paper/2606.31159/) (2026)
- [AgenTag: Attribution of AI Coding Agents from Behavioral Fingerprints](https://mlsanity.com/paper/2608.00966/) (2026)
- [Engineering Signals of Human-AI Collaboration in the Agentic Coding Era: A Longitudinal Analysis of 33,228 Pull Requests from vLLM and SGLang with Implications for Biomedical AI Agents and Bioinformatics Pipeline Developmen](https://mlsanity.com/paper/2608.13884/) (2026)
- [Concepts for Securing Agentic AI Coding and the Terok Environment](https://mlsanity.com/paper/2608.22930/) (2026)

## Abstract (arXiv)

Autonomous coding agents are increasingly deployed as AI teammates in modern software engineering, independently authoring pull requests (PRs) that modify production code at scale. This study aims to systematically characterize how autonomous coding agents contribute to software security in practice, how these security-related contributions are reviewed and accepted, and which observable signals are associated with PR rejection. We conduct a large-scale empirical analysis of agent-authored PRs using the AIDev dataset, comprising of over 33,000 curated PRs from popular GitHub repositories. Security-relevant PRs are identified using a keyword filtering strategy, followed by manual validation, resulting in 1,293 confirmed security-related agentic-PRs. We then analyze prevalence, acceptance outcomes, and review latency across autonomous agents, programming ecosystems, and types of code changes. Moreover, we apply qualitative open coding to identify recurring security-related actions and underlying intents, and examine review metadata to identify early signals associated with PR rejection. Security-related Agentic-PRs constitute a meaningful share of agent activity (approximately 4%). Rather than focusing solely on narrow vulnerability fixes, agents most frequently perform supportive security hardening activities, including testing, documentation, configuration, and improved error handling. Compared to non-security PRs, security-related Agentic-PRs exhibit lower merge rates and longer review latency, reflecting heightened human scrutiny, with variation across agents and programming ecosystems. PR rejection is more strongly associated with PR complexity and verbosity than with explicit security topics.