When AI Agents Touch CI/CD Configurations: Frequency and Success
Not on the map. Why?
Abstract (arXiv)
AI agents are increasingly used in software development, yet their interaction with CI/CD configurations is not well studied. We analyze 8,031 agentic pull requests (PRs) from 1,605 GitHub repositories where AI agents touch YAML configurations. CI/CD configuration files account for 3.25% of agent changes, varying by agent (Devin: 4.83%, Codex: 2.01%, p < 0.001). When agents modify CI/CD, 96.77% target GitHub Actions. Agentic PRs with CI/CD changes merge slightly less often than others (67.77% vs. 71.80%), except for Copilot, whose CI/CD changes merge 15.63 percentage points more often. Across 99,930 workflow runs, build success rates are comparable for CI/CD and non-CI/CD changes (75.59% vs. 74.87%), though three agents show significantly higher success when modifying CI/CD. These results show that AI agents rarely modify CI/CD and focus mostly on GitHub Actions, yet their configuration changes are as reliable as regular code. Copilot's strong CI/CD performance despite lower acceptance suggests emerging configuration specialization, with implications for agent training and DevOps automation.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Jan 2026.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub
- 2026Reliability of AI Bots Footprints in GitHub Actions CI/CD Workflows
- 2026Why Are AI Agent Involved Pull Requests (Fix-Related) Remain Unmerged? An Empirical Study
- 2025On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub
- 2026How AI Coding Agents Modify Code: A Large-Scale Study of GitHub Pull Requests
- 2026Safer Builders, Risky Maintainers: A Comparative Study of Breaking Changes in Human vs Agentic PRs
- 2026Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests
- 2026On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents