# LLMs' Reshaping of People, Processes, Products, and Society in Software Development: A Comprehensive Exploration with Early Adopters

arXiv: https://arxiv.org/abs/2503.05012 · HTML: https://mlsanity.com/paper/2503.05012/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 6 Mar 2025.

## Builds on (most cited first, 14 of 14)

- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Process](https://mlsanity.com/paper/2110.10234/) (2021)
- [Exploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Study](https://mlsanity.com/paper/2309.08221/) (2023)
- [On the Evaluation of Large Language Models in Unit Test Generation](https://mlsanity.com/paper/2406.18181/) (2024)
- [Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concerns](https://mlsanity.com/paper/2405.06371/) (2024)
- [Exploring the Design Space of Cognitive Engagement Techniques with AI-Generated Code for Enhanced Learning](https://mlsanity.com/paper/2410.08922/) (2024)
- [How to Refactor this Code? An Exploratory Study on Developer-ChatGPT Refactoring Conversations](https://mlsanity.com/paper/2402.06013/) (2024)
- [Developer Experiences with a Contextualized AI Coding Assistant: Usability, Expectations, and Outcomes](https://mlsanity.com/paper/2311.18452/) (2023)
- [The GenUI Study: Exploring the Design of Generative UI Tools to Support UX Practitioners and Beyond](https://mlsanity.com/paper/2501.13145/) (2025)
- [Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions](https://mlsanity.com/paper/2502.14202/) (2025)
- [An Exploratory Study on Upper-Level Computing Students' Use of Large Language Models as Tools in a Semester-Long Project](https://mlsanity.com/paper/2403.18679/) (2024)
- [What Skills Do You Need When Developing Software Using ChatGPT? (Discussion Paper)](https://mlsanity.com/paper/2310.05998/) (2023)
- [Understanding AR Activism: An Interview Study with Creators of Augmented Reality Experiences for Social Change](https://mlsanity.com/paper/2204.09821/) (2022)

## Built on by (most cited first, 8 of 8)

- [Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI](https://mlsanity.com/paper/2505.19443/) (2025)
- [Benchmarking Correctness and Security in Multi-Turn Code Generation](https://mlsanity.com/paper/2510.13859/) (2025)
- [Novice Developers' Perspectives on Adopting LLMs for Software Development: A Systematic Literature Review](https://mlsanity.com/paper/2503.07556/) (2025)
- [Learning to Generate Secure Code via Token-Level Rewards](https://mlsanity.com/paper/2602.23407/) (2026)
- [Mitigating Prompt-Induced Cognitive Biases in General-Purpose AI for Software Engineering](https://mlsanity.com/paper/2604.16756/) (2026)
- [Single Conversation Methodology: A Human-Centered Protocol for AI-Assisted Software Development](https://mlsanity.com/paper/2507.12665/) (2025)
- [Simulating Teams with LLM Agents: Interactive 2D Environments for Studying Human-AI Dynamics](https://mlsanity.com/paper/2510.08242/) (2025)
- [From Adoption to Deployment: A Qualitative Study on AI Integration in Software Development Practice](https://mlsanity.com/paper/2607.16660/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are rapidly reshaping software development, but their impact across the software development lifecycle is underexplored. Existing work focuses on isolated activities such as code generation or testing, leaving open questions about how LLMs affect developers, processes, products, and the software ecosystem. We address this gap through semi-structured interviews with sixteen early-adopter software professionals who integrated LLM-based tools into their day-to-day work in early to mid-2023. We treat these interviews as early empirical evidence and compare participants' accounts with recent work on LLMs in software engineering, noting which early patterns persist or shift. Using thematic analysis, we organize findings around four dimensions: people, process, product, and society. Developers reported substantial productivity gains from reducing routine tasks, streamlining search, and accelerating debugging, but also described a productivity-quality paradox: they often discarded generated code and shifted effort from writing to critically evaluating and integrating it. LLM use was highly phase-dependent, with strong uptake in implementation and debugging but limited influence on requirements gathering and other collaborative work. Participants developed new competencies to use LLMs effectively, including prompt engineering strategies, layered verification, and secure integration to protect proprietary data. They anticipated changes in hiring expectations, team practices, and computing education, while emphasizing that human judgment and foundational software engineering skills remain essential. Our findings, later echoed in large-scale quantitative studies, offer actionable implications for developers, organizations, educators, and tool designers seeking to integrate LLMs responsibly into software practice today.