mlsanityData to 14 Oct 2026

Beyond Simpson's Paradox: A Cascade of Confounders in AI Agent Pull-Request Co-Authorship

arXiv 2606.22711 ↗First posted 21 Jun 2026cs.SE cs.AIcited by 1

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Pooled across five AI coding agents, pull requests (PRs) with a human Co-Authored-By trailer merge less often than purely-autonomous ones (53.8% vs. 79.8%) -- yet this aggregate finding is a textbook Simpson's Paradox. Stratifying 33,596 PRs from the AIDev dataset by agent identity reverses the conclusion: Copilot and Devin show large positive within-agent gaps (+41.2 and +33.5 pp, both p<0.001), while Cursor, Claude Code, and Codex show small effects whose cross-sectional 95% CIs span zero. The paradox is driven entirely by agent composition: Codex, which dominates 64.9% of the dataset, achieves high merge rates while rarely using co-authorship. But Simpson's Paradox is only the first layer of a cascade of confounders: within-repo controls eliminate Devin's gap (+33.5 to +1.6 pp, p=0.73); a commit-count control further halves Copilot's within-repo gap (+36.2 to +24.4 pp); restricted to multi-commit PRs, the Copilot within-repo effect dissolves to +4.8 pp (p=0.59). No agent retains a clear co-authorship effect once both repository selection and PR structure are controlled. Our findings caution against reporting agent-pooled statistics without stratification and demonstrate that cross-sectional co-authorship associations are largely selection and PR-structure artefacts rather than evidence of a causal benefit.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 3 of its 14 references that are in the map. First posted on arXiv 21 Jun 2026.

Builds on

3 of 14 references.

References · 3

  1. 2022GitHub Copilot AI pair programmer: Asset or Liability?cited by 152
  2. 2021Quality Gatekeepers: Investigating the Effects ofCode Review Bots on Pull Request Activitiescited by 8
  3. 2021Don't Disturb Me: Challenges of Interacting with SoftwareBots on Open Source Software Projectscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Govern the Repository, Not the Agent: Measuring Ecosystem-Level Risk in AI-Native Software
  2. 2026Why Are Agentic Pull Requests Merged or Rejected? An Empirical Study
  3. 2026Who Finishes the Job? A Study of Follow-Up Fixes and Commit Authorship on AI Coding Agent Pull Requests
  4. 2026Let's Make Every Pull Request Meaningful: An Empirical Analysis of Developer and Agentic Pull Requests
  5. 2026Humans Integrate, Agents Fix: How Agent-Authored Pull Requests Are Referenced in Practice
  6. 2026Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub
  7. 2026AgenTag: Attribution of AI Coding Agents from Behavioral Fingerprints
  8. 2026Analyzing Message-Code Inconsistency in AI Coding Agent-Authored Pull Requests