# DevGPT: Studying Developer-ChatGPT Conversations

arXiv: https://arxiv.org/abs/2309.03914 · HTML: https://mlsanity.com/paper/2309.03914/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2023.

## Built on by (most cited first, 12 of 12)

- [The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmers](https://mlsanity.com/paper/2404.02806/) (2024)
- [How Developers Interact with AI: A Taxonomy of Human-AI Collaboration in Software Engineering](https://mlsanity.com/paper/2501.08774/) (2025)
- [ProPerSim: Developing Proactive and Personalized AI Assistants through User-Assistant Simulation](https://mlsanity.com/paper/2509.21730/) (2025)
- [The Fault in our Stars: Quality Assessment of Code Generation Benchmarks](https://mlsanity.com/paper/2404.10155/) (2024)
- [Generative AI and Empirical Software Engineering: A Paradigm Shift](https://mlsanity.com/paper/2502.08108/) (2025)
- [Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions](https://mlsanity.com/paper/2502.14202/) (2025)
- [Promises, Perils, and (Timely) Heuristics for Mining Coding Agent Activity](https://mlsanity.com/paper/2601.18345/) (2026)
- [AgenticFlict: A Large-Scale Dataset of Merge Conflicts in AI Coding Agent Pull Requests on GitHub](https://mlsanity.com/paper/2604.03551/) (2026)
- [Do Prompt Patterns Affect Code Quality? A First Empirical Assessment of ChatGPT-Generated Code](https://mlsanity.com/paper/2504.13656/) (2025)
- [Mitigating Prompt-Induced Cognitive Biases in General-Purpose AI for Software Engineering](https://mlsanity.com/paper/2604.16756/) (2026)
- ["Nothing to See Here'': Unintended Disclosure through Revision Traces of LLM Deliverables](https://mlsanity.com/paper/2609.35408/) (2026)
- [Why Agentic Workflows Fail: A Taxonomy Derived from Literature and Empirical Data](https://mlsanity.com/paper/2610.17418/) (2026)

## Abstract (arXiv)

This paper introduces DevGPT, a dataset curated to explore how software developers interact with ChatGPT, a prominent large language model (LLM). The dataset encompasses 29,778 prompts and responses from ChatGPT, including 19,106 code snippets, and is linked to corresponding software development artifacts such as source code, commits, issues, pull requests, discussions, and Hacker News threads. This comprehensive dataset is derived from shared ChatGPT conversations collected from GitHub and Hacker News, providing a rich resource for understanding the dynamics of developer interactions with ChatGPT, the nature of their inquiries, and the impact of these interactions on their work. DevGPT enables the study of developer queries, the effectiveness of ChatGPT in code generation and problem solving, and the broader implications of AI-assisted programming. By providing this dataset, the paper paves the way for novel research avenues in software engineering, particularly in understanding and improving the use of LLMs like ChatGPT by developers.