# Adoption and Impact of Command-Line AI Coding Agents: A Study of Microsoft's Early 2026 Rollout of Claude Code and GitHub Copilot CLI

arXiv: https://arxiv.org/abs/2607.01418 · HTML: https://mlsanity.com/paper/2607.01418/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 23 of its 45 references that are in the map.
- First posted on arXiv 1 Jul 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 23 of 23)

- [The Impact of AI on Developer Productivity: Evidence from GitHub Copilot](https://mlsanity.com/paper/2302.06590/) (2023)
- [Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity](https://mlsanity.com/paper/2507.09089/) (2025)
- [Productivity Assessment of Neural Code Completion](https://mlsanity.com/paper/2205.06537/) (2022)
- [Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming](https://mlsanity.com/paper/2210.14306/) (2022)
- [Agentic Much? Adoption of Coding Agents on GitHub](https://mlsanity.com/paper/2601.18341/) (2026)
- [Speed at the Cost of Quality: How Cursor AI Increases Short-Term Velocity and Long-Term Complexity in Open-Source Projects](https://mlsanity.com/paper/2511.04427/) (2025)
- [How much does AI impact development speed? An enterprise-based randomized controlled trial](https://mlsanity.com/paper/2410.12944/) (2024)
- [Who is using AI to code? Global diffusion and impact of generative AI](https://mlsanity.com/paper/2506.08945/) (2025)
- [The Impact of Large Language Models on Open-source Innovation: Evidence from GitHub Copilot](https://mlsanity.com/paper/2409.08379/) (2024)
- [The Impact of Generative AI on Collaborative Open-Source Software Development: Evidence from GitHub Copilot](https://mlsanity.com/paper/2410.02091/) (2024)
- [Why AI Agents Still Need You: Findings from Developer-Agent Collaborations in the Wild](https://mlsanity.com/paper/2506.12347/) (2025)
- [AI IDEs or Autonomous Agents? Measuring the Impact of Coding Agents on Software Development](https://mlsanity.com/paper/2601.13597/) (2026)
- [The Effects of GitHub Copilot on Computing Students' Programming Effectiveness, Efficiency, and Processes in Brownfield Programming Tasks](https://mlsanity.com/paper/2506.10051/) (2025)
- [Dear Diary: A randomized controlled trial of Generative AI coding tools in the workplace](https://mlsanity.com/paper/2410.18334/) (2024)
- [Intuition to Evidence: Measuring AI's True Impact on Developer Productivity](https://mlsanity.com/paper/2509.19708/) (2025)
- [Developer Productivity With and Without GitHub Copilot: A Longitudinal Mixed-Methods Case Study](https://mlsanity.com/paper/2509.20353/) (2025)
- [The Impact of LLM-Assistants on Software Developer Productivity: A Systematic Review and Mapping Study](https://mlsanity.com/paper/2507.03156/) (2025)
- [Developers' Experience with Generative AI -- First Insights from an Empirical Mixed-Methods Field Study](https://mlsanity.com/paper/2512.19926/) (2025)
- [The Fast and Spurious: Developer Productivity with GenAI](https://mlsanity.com/paper/2510.24265/) (2025)
- [The Heterogeneous Productivity Effects of Generative AI](https://mlsanity.com/paper/2403.01964/) (2024)
- [The State of Generative AI in Software Development: Insights from Literature and a Developer Survey](https://mlsanity.com/paper/2603.16975/) (2026)
- [Impacts of Generative AI on Agile Teams' Productivity: A Multi-Case Longitudinal Study](https://mlsanity.com/paper/2602.13766/) (2026)
- [GitHub Copilot and Developer Productivity: An Observational Dose-Response Analysis](https://mlsanity.com/paper/2606.00438/) (2026)

## Built on by (most cited first, 1 of 1)

- [RECLAIM: Can Agents Reproduce the Claims of Machine Learning Papers?](https://mlsanity.com/paper/2609.28850/) (2026)

## Abstract (arXiv)

Organizations rolling out agentic command line tools like Anthropic's Claude Code and GitHub's Copilot CLI need to know who will try them, who will keep using them, and whether the tools produce enough output to justify their cost. At organizational scale, token spend can run into millions of dollars annually, so misreading adoption, retention, or impact can make a rollout expensive without changing engineering velocity. Studying tens of thousands of engineers at Microsoft over its early-2026 rollout, we find that first use spread primarily through social networks, retention was associated more with engineers' coding activity than with demographics, and adopters merged roughly 24% more pull requests than they would have otherwise. We use merged pull requests as our proxy for output -- acknowledging that a merged PR is not the same as the value it delivers -- and the lift persists across our four-month window. These results suggest that CLI coding agents are neither uniformly adopted nor mere novelty effects and that organizations should treat visible peer use as central to rollout strategy.