mlsanityData to 7 Oct 2026

Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHub

arXiv 2601.00477 ↗First posted 1 Jan 2026cs.CR cs.SEcited by 6

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Autonomous coding agents are increasingly deployed as AI teammates in modern software engineering, independently authoring pull requests (PRs) that modify production code at scale. This study aims to systematically characterize how autonomous coding agents contribute to software security in practice, how these security-related contributions are reviewed and accepted, and which observable signals are associated with PR rejection. We conduct a large-scale empirical analysis of agent-authored PRs using the AIDev dataset, comprising of over 33,000 curated PRs from popular GitHub repositories. Security-relevant PRs are identified using a keyword filtering strategy, followed by manual validation, resulting in 1,293 confirmed security-related agentic-PRs. We then analyze prevalence, acceptance outcomes, and review latency across autonomous agents, programming ecosystems, and types of code changes. Moreover, we apply qualitative open coding to identify recurring security-related actions and underlying intents, and examine review metadata to identify early signals associated with PR rejection. Security-related Agentic-PRs constitute a meaningful share of agent activity (approximately 4%). Rather than focusing solely on narrow vulnerability fixes, agents most frequently perform supportive security hardening activities, including testing, documentation, configuration, and improved error handling. Compared to non-security PRs, security-related Agentic-PRs exhibit lower merge rates and longer review latency, reflecting heightened human scrutiny, with variation across agents and programming ecosystems. PR rejection is more strongly associated with PR complexity and verbosity than with explicit security topics.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 32 of its 60 references that are in the map. First posted on arXiv 1 Jan 2026.

Builds on

32 of 60 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025ProofWright: Towards Agentic Formal Verification of CUDA15 Nov
  2. 2025Agentsway -- Software Development Methodology for AI Agents-based Teams26 Oct
  3. 2025Toward Agentic Software Engineering Beyond Code: Framing Vision, Values, and Vocabulary22 Oct
  4. 2025An Empirical Study of Testing Practices in Open Source AI Agent Frameworks and Agentic Applications23 Sep
  5. 2025On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub18 Sep
  6. 2025Agentic Software Engineering: Foundational Pillars and a Research Roadmap7 Sep
  7. 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities15 Aug
  8. 2025Taming Uncertainty via Automation: Observing, Analyzing, and Optimizing Agentic AI Systems15 Jul
  9. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI26 May
  10. 2025Facilitating Trustworthy Human-Agent Collaboration in LLM-based Multi-Agent System oriented Software Engineering7 May
  11. 2024Experimenting with Multi-Agent Software Development: Towards a Unified Platform8 Jun
  12. 2024CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology2 Feb

Other references · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  6. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  7. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  8. 2016Bag of Tricks for Efficient Text Classificationcited by 869
  9. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 417
  10. 2023Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Modelscited by 130
  11. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  12. 2023SALLM: Security Assessment of Generated Codecited by 35
8 more
  1. 2025Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Taskscited by 21
  2. 2023FRANC: A Lightweight Framework for High-Quality Code Generationcited by 14
  3. 2019Does Code Quality Affect Pull Request Acceptance? An empirical studycited by 9
  4. 2021Pull Request Latency Explained: An Empirical Overviewcited by 6
  5. 2024ChatCollab: Exploring Collaboration Between Humans and AI Agents in Software Teamscited by 6
  6. 2025Formalizing the Safety, Security, and Functional Properties of Agentic AI Systemscited by 5
  7. 2021Pull Request Decision Explained: An Empirical Overviewcited by 2
  8. 2024A Mixed-Methods Study of Open-Source Software Maintainers On Vulnerability Management and Platform Security Featurescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Insights into Security-Related AI-Generated Pull Requests
  2. 2026Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents
  3. 2026Safer Builders, Risky Maintainers: A Comparative Study of Breaking Changes in Human vs Agentic PRs
  4. 2026Investigating Autonomous Agent Contributions in the Wild: Activity Patterns and Code Change over Time
  5. 2026Not All Agents Are Equal: Code Quality and Post-Merge Maintenance Across Five Autonomous Coding Agents in the Wild
  6. 2026Why Agentic-PRs Get Rejected: A Comparative Study of Coding Agents
  7. 2026How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests
  8. 2026Fingerprinting AI Coding Agents on GitHub