Conversational AI as a Coding Assistant: Understanding Programmers' Interactions with and Expectations from Large Language Models for Coding
Not on the map. Why?
Abstract (arXiv)
Conversational AI interfaces powered by large language models (LLMs) are increasingly used as coding assistants. However, questions remain about how programmers interact with LLM-based conversational agents, the challenges they encounter, and the factors influencing adoption. This study investigates programmers' usage patterns, perceptions, and interaction strategies when engaging with LLM-driven coding assistants. Through a survey, participants reported both the benefits, such as efficiency and clarity of explanations, and the limitations, including inaccuracies, lack of contextual awareness, and concerns about over-reliance. Notably, some programmers actively avoid LLMs due to a preference for independent learning, distrust in AI-generated code, and ethical considerations. Based on our findings, we propose design guidelines for improving conversational coding assistants, emphasizing context retention, transparency, multimodal support, and adaptability to user preferences. These insights contribute to the broader understanding of how LLM-based conversational agents can be effectively integrated into software development workflows while addressing adoption barriers and enhancing usability.
Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 10 of its 42 references that are in the map. First posted on arXiv 14 Mar 2025.
Builds on
10 of 42 references.
References · 10
- 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
- 2023Using an LLM to Help With Code Understandingcited by 222
- 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 216
- 2022Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generationcited by 97
- 2023Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programmingcited by 87
- 2023A Survey on Large Language Models for Software Engineeringcited by 58
- 2024An Empirical Study on Usage and Perceptions of LLMs in a Software Engineering Projectcited by 28
- 2024Examining the Use and Impact of an AI Code Assistant on Developer Productivity and Experience in the Enterprisecited by 17
- 2024A Study on Developer Behaviors for Validating and Repairing LLM-Generated Code Using Eye Tracking and IDE Actionscited by 12
- 2024Understanding the Human-LLM Dynamic: A Literature Survey of LLM Use in Programming Taskscited by 9
Built on by
Citing papers per year: 2025: 8; 2026: 3.
11 papers.
- 2025Agent READMEs: An Empirical Study of Context Files for Agentic Codingcited by 21
- 2025A Study of LLMs' Preferences for Libraries and Programming Languagescited by 20
- 2026OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMscited by 18
- 2025Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025cited by 17
- 2025On the Use of Agentic Coding Manifests: An Empirical Study of Claude Codecited by 14
- 2025Vibe Coding as a Reconfiguration of Intent Mediation in Software Development: Definition, Implications, and Research Agendacited by 13
- 2025ProPerSim: Developing Proactive and Personalized AI Assistants through User-Assistant Simulationcited by 10
- 2025Vibe Coding in Product Teams: Reconfiguring AI-Assisted Workflows, Prototyping, and Collaborationcited by 3
- 2025MermaidSeqBench: An Evaluation Benchmark for NL-to-Mermaid Sequence Diagram Generationcited by 3
- 2026Agentic AI in Industry: Adoption Level and Deployment Barrierscited by 3
- 2026Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?cited by 0
- 2026Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?23 Jul 2026
- 2026Agentic AI in Industry: Adoption Level and Deployment Barriers14 May 2026
- 2026OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMs4 Jan 2026
- 2025Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 202516 Dec 2025
- 2025MermaidSeqBench: An Evaluation Benchmark for NL-to-Mermaid Sequence Diagram Generation18 Nov 2025
- 2025Agent READMEs: An Empirical Study of Context Files for Agentic Coding17 Nov 2025
- 2025ProPerSim: Developing Proactive and Personalized AI Assistants through User-Assistant Simulation26 Sep 2025
- 2025On the Use of Agentic Coding Manifests: An Empirical Study of Claude Code18 Sep 2025
- 2025Vibe Coding in Product Teams: Reconfiguring AI-Assisted Workflows, Prototyping, and Collaboration12 Sep 2025
- 2025Vibe Coding as a Reconfiguration of Intent Mediation in Software Development: Definition, Implications, and Research Agenda29 Jul 2025
- 2025A Study of LLMs' Preferences for Libraries and Programming Languages21 Mar 2025