mlsanityData to 14 Oct 2026

Detecting AI Coding Agents in Open Source: A Validated Multi-Method Census of 180 Million Repositories

arXiv 2606.24429 ↗First posted 23 Jun 2026cs.SE cs.AIcited by 0

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Generative AI coding agents are entering the open-source supply chain, yet their diverse and often invisible traces leave their prevalence poorly understood. We introduce a multi-layered detection framework that integrates configuration-file scanning, commit-message analysis, author-identity matching, and bot-signature lookup across World of Code (180M+ Git repositories), classifying agent traces into four behavioral types. No single method captures more than a fraction of activity: multi-method detection identifies 850,157 Claude Code commits in one snapshot, of which bot-account lookup_the signal most adoption studies rely on_recovers only 28,154 (3.3%), a 30x relative-recall gap, so single-signal prevalence estimates are biased low by at least this factor. Every detection pattern is hand-validated (495 labels) with per-cell precision and Wilson confidence intervals. Across snapshots from December 2024 to April 2026, commit-attributed agents generate over 320,000 commits per month; Claude Code leads (886,122 commits across 17,295 projects) and dominates silent, configuration-file-only adoption (21,078 projects). Compared against an independent pull-request census (AIDev), the two channels capture nearly disjoint agent populations_a PR census misses 79% of commit-detected Claude Code adopters and essentially all Codex adopters_and different kinds of work: PR-deployed cloud agents (Codex, Cursor) surface as feature work, while commit-deployed in-editor agents (Claude Code, OpenHands, Aider) surface as maintenance. The observed work profile follows deployment and detection mode rather than the tool itself, so no single channel is representative.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 13 of its 35 references that are in the map. First posted on arXiv 23 Jun 2026.

Builds on

13 of 35 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Composer 2 Technical Report25 Mar
  2. 2026AIDev: Studying AI Coding Agents on GitHub9 Feb
  3. 2026Agentic Much? Adoption of Coding Agents on GitHub26 Jan
  4. 2025Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projects6 Nov
  5. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul

Other references · 8

  1. 2023The Impact of AI on Developer Productivity: Evidence from GitHub Copilotcited by 274
  2. 2023Zero-Shot Detection of Machine-Generated Codescited by 24
  3. 2020Detecting and Characterizing Bots that Commit Codecited by 10
  4. 2020A ground-truth dataset and classification model for detecting bots in GitHub issue and PR commentscited by 10
  5. 2020World of Code: Enabling a Research Workflow for Mining and Analyzing the Universe of Open Source VCS datacited by 7
  6. 2020A Complete Set of Related Git Repositories Identified via Community Detection Approaches Based on Shared Commitscited by 3
  7. 2020A Dataset and an Approach for Identity Resolution of 38 Million Author IDs extracted from 2B Git Commitscited by 3
  8. 2025Detecting LLM-generated Code with Subtle Modification by Adversarial Trainingcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Investigating Autonomous Agent Contributions in the Wild: Activity Patterns and Code Change over Time
  2. 2026Fingerprinting AI Coding Agents on GitHub
  3. 2026How AI Coding Agents Modify Code: A Large-Scale Study of GitHub Pull Requests
  4. 2026AgenTag: Attribution of AI Coding Agents from Behavioral Fingerprints
  5. 2026AgentLogs: A Dataset for Opening the Black Box of GitHub's Cloud Agent
  6. 2025On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub
  7. 2026A Task-Level Evaluation of AI Agents in Open-Source Projects
  8. 2026Not All Agents Are Equal: Code Quality and Post-Merge Maintenance Across Five Autonomous Coding Agents in the Wild