# Unveiling ChatGPT's Usage in Open Source Projects: A Mining-based Study

arXiv: https://arxiv.org/abs/2402.16480 · HTML: https://mlsanity.com/paper/2402.16480/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Feb 2024.

## Built on by (most cited first, 16 of 16)

- [On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub](https://mlsanity.com/paper/2509.14745/) (2025)
- [Agentic Much? Adoption of Coding Agents on GitHub](https://mlsanity.com/paper/2601.18341/) (2026)
- [Agentic Refactoring: An Empirical Study of AI Coding Agents](https://mlsanity.com/paper/2511.04824/) (2025)
- [Decoding the Configuration of AI Coding Agents: Insights from Claude Code Projects](https://mlsanity.com/paper/2511.09268/) (2025)
- [Students' Perceptions and Use of Generative AI Tools for Programming Across Different Computing Courses](https://mlsanity.com/paper/2410.06865/) (2024)
- [Promises, Perils, and (Timely) Heuristics for Mining Coding Agent Activity](https://mlsanity.com/paper/2601.18345/) (2026)
- [Are Coding Agents Generating Over-Mocked Tests? An Empirical Study](https://mlsanity.com/paper/2602.00409/) (2026)
- [Investigating Autonomous Agent Contributions in the Wild: Activity Patterns and Code Change over Time](https://mlsanity.com/paper/2604.00917/) (2026)
- [How Do Agents Perform Code Optimization? An Empirical Study](https://mlsanity.com/paper/2512.21757/) (2025)
- [Chain of Targeted Verification Questions to Improve the Reliability of Code Generated by LLMs](https://mlsanity.com/paper/2405.13932/) (2024)
- [Do AI Coding Agents Log Like Humans? An Empirical Study](https://mlsanity.com/paper/2604.09409/) (2026)
- [Towards Refining Developer Questions using LLM-Based Named Entity Recognition for Developer Chatroom Conversations](https://mlsanity.com/paper/2503.00673/) (2025)
- [CodeGenLink: A Tool to Find the Likely Origin and License of Automatically Generated Code](https://mlsanity.com/paper/2510.01077/) (2025)
- [The Invisible Hand of AI Libraries Shaping Open Source Projects and Communities](https://mlsanity.com/paper/2601.01944/) (2026)
- [Do AI Agents Really Improve Code Readability?](https://mlsanity.com/paper/2603.13723/) (2026)
- [On the Maintenance and Co-evolution of Agent Plugins: An Empirical Study of Claude Code Plugin Marketplaces](https://mlsanity.com/paper/2608.28497/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have gained significant attention in the software engineering community. Nowadays developers have the possibility to exploit these models through industrial-grade tools providing a handy interface toward LLMs, such as OpenAI's ChatGPT. While the potential of LLMs in assisting developers across several tasks has been documented in the literature, there is a lack of empirical evidence mapping the actual usage of LLMs in software projects. In this work, we aim at filling such a gap. First, we mine 1,501 commits, pull requests (PRs), and issues from open-source projects by matching regular expressions likely to indicate the usage of ChatGPT to accomplish the task. Then, we manually analyze these instances, discarding false positives (i.e., instances in which ChatGPT was mentioned but not actually used) and categorizing the task automated in the 467 true positive instances (165 commits, 159 PRs, 143 issues). This resulted in a taxonomy of 45 tasks which developers automate via ChatGPT. The taxonomy, accompanied with representative examples, provides (i) developers with valuable insights on how to exploit LLMs in their workflow and (ii) researchers with a clear overview of tasks that, according to developers, could benefit from automated solutions.