mlsanityData to 14 Oct 2026

LLMs' Reshaping of People, Processes, Products, and Society in Software Development: A Comprehensive Exploration with Early Adopters

arXiv 2503.05012 ↗First posted 6 Mar 2025cs.SE cs.AI cs.HCcited by 8

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are rapidly reshaping software development, but their impact across the software development lifecycle is underexplored. Existing work focuses on isolated activities such as code generation or testing, leaving open questions about how LLMs affect developers, processes, products, and the software ecosystem. We address this gap through semi-structured interviews with sixteen early-adopter software professionals who integrated LLM-based tools into their day-to-day work in early to mid-2023. We treat these interviews as early empirical evidence and compare participants' accounts with recent work on LLMs in software engineering, noting which early patterns persist or shift. Using thematic analysis, we organize findings around four dimensions: people, process, product, and society. Developers reported substantial productivity gains from reducing routine tasks, streamlining search, and accelerating debugging, but also described a productivity-quality paradox: they often discarded generated code and shifted effort from writing to critically evaluating and integrating it. LLM use was highly phase-dependent, with strong uptake in implementation and debugging but limited influence on requirements gathering and other collaborative work. Participants developed new competencies to use LLMs effectively, including prompt engineering strategies, layered verification, and secure integration to protect proprietary data. They anticipated changes in hiring expectations, team practices, and computing education, while emphasizing that human judgment and foundational software engineering skills remain essential. Our findings, later echoed in large-scale quantitative studies, offer actionable implications for developers, organizations, educators, and tool designers seeking to integrate LLMs responsibly into software practice today.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 14 of its 34 references that are in the map. First posted on arXiv 6 Mar 2025.

Builds on

14 of 34 references.

References · 14

  1. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
  2. 2023Large Language Models for Software Engineering: A Systematic Literature Reviewcited by 399
  3. 2021Collaboration Challenges in Building ML-Enabled Systems: Communication, Documentation, Engineering, and Processcited by 56
  4. 2023Exploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Studycited by 44
  5. 2024On the Evaluation of Large Language Models in Unit Test Generationcited by 34
  6. 2024Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concernscited by 28
  7. 2024Exploring the Design Space of Cognitive Engagement Techniques with AI-Generated Code for Enhanced Learningcited by 16
  8. 2024How to Refactor this Code? An Exploratory Study on Developer-ChatGPT Refactoring Conversationscited by 12
  9. 2023Developer Experiences with a Contextualized AI Coding Assistant: Usability, Expectations, and Outcomescited by 9
  10. 2025The GenUI Study: Exploring the Design of Generative UI Tools to Support UX Practitioners and Beyondcited by 9
  11. 2025Do LLMs Consider Security? An Empirical Study on Responses to Programming Questionscited by 9
  12. 2024An Exploratory Study on Upper-Level Computing Students' Use of Large Language Models as Tools in a Semester-Long Projectcited by 6
2 more
  1. 2023What Skills Do You Need When Developing Software Using ChatGPT? (Discussion Paper)cited by 3
  2. 2022Understanding AR Activism: An Interview Study with Creators of Augmented Reality Experiences for Social Changecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 3.

8 papers.