# Conversational AI as a Coding Assistant: Understanding Programmers' Interactions with and Expectations from Large Language Models for Coding

arXiv: https://arxiv.org/abs/2503.16508 · HTML: https://mlsanity.com/paper/2503.16508/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 42 references that are in the map.
- First posted on arXiv 14 Mar 2025.

## Builds on (most cited first, 10 of 10)

- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [Using an LLM to Help With Code Understanding](https://mlsanity.com/paper/2307.08177/) (2023)
- [Grounded Copilot: How Programmers Interact with Code-Generating Models](https://mlsanity.com/paper/2206.15000/) (2022)
- [Programming Is Hard -- Or at Least It Used to Be: Educational Opportunities And Challenges of AI Code Generation](https://mlsanity.com/paper/2212.01020/) (2022)
- [Studying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming](https://mlsanity.com/paper/2302.07427/) (2023)
- [A Survey on Large Language Models for Software Engineering](https://mlsanity.com/paper/2312.15223/) (2023)
- [An Empirical Study on Usage and Perceptions of LLMs in a Software Engineering Project](https://mlsanity.com/paper/2401.16186/) (2024)
- [Examining the Use and Impact of an AI Code Assistant on Developer Productivity and Experience in the Enterprise](https://mlsanity.com/paper/2412.06603/) (2024)
- [A Study on Developer Behaviors for Validating and Repairing LLM-Generated Code Using Eye Tracking and IDE Actions](https://mlsanity.com/paper/2405.16081/) (2024)
- [Understanding the Human-LLM Dynamic: A Literature Survey of LLM Use in Programming Tasks](https://mlsanity.com/paper/2410.01026/) (2024)

## Built on by (most cited first, 11 of 11)

- [Agent READMEs: An Empirical Study of Context Files for Agentic Coding](https://mlsanity.com/paper/2511.12884/) (2025)
- [OpenRT: An Open-Source Red Teaming Framework for Multimodal LLMs](https://mlsanity.com/paper/2601.01592/) (2026)
- [A Study of LLMs' Preferences for Libraries and Programming Languages](https://mlsanity.com/paper/2503.17181/) (2025)
- [Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025](https://mlsanity.com/paper/2512.14012/) (2025)
- [On the Use of Agentic Coding Manifests: An Empirical Study of Claude Code](https://mlsanity.com/paper/2509.14744/) (2025)
- [Vibe Coding as a Reconfiguration of Intent Mediation in Software Development: Definition, Implications, and Research Agenda](https://mlsanity.com/paper/2507.21928/) (2025)
- [ProPerSim: Developing Proactive and Personalized AI Assistants through User-Assistant Simulation](https://mlsanity.com/paper/2509.21730/) (2025)
- [Vibe Coding in Product Teams: Reconfiguring AI-Assisted Workflows, Prototyping, and Collaboration](https://mlsanity.com/paper/2509.10652/) (2025)
- [Agentic AI in Industry: Adoption Level and Deployment Barriers](https://mlsanity.com/paper/2605.14675/) (2026)
- [MermaidSeqBench: An Evaluation Benchmark for NL-to-Mermaid Sequence Diagram Generation](https://mlsanity.com/paper/2511.14967/) (2025)
- [Bespoke Visual Assistance: What and How do Blind and Low-Vision People Create with Agentic Programming?](https://mlsanity.com/paper/2607.21760/) (2026)

## Abstract (arXiv)

Conversational AI interfaces powered by large language models (LLMs) are increasingly used as coding assistants. However, questions remain about how programmers interact with LLM-based conversational agents, the challenges they encounter, and the factors influencing adoption. This study investigates programmers' usage patterns, perceptions, and interaction strategies when engaging with LLM-driven coding assistants. Through a survey, participants reported both the benefits, such as efficiency and clarity of explanations, and the limitations, including inaccuracies, lack of contextual awareness, and concerns about over-reliance. Notably, some programmers actively avoid LLMs due to a preference for independent learning, distrust in AI-generated code, and ethical considerations. Based on our findings, we propose design guidelines for improving conversational coding assistants, emphasizing context retention, transparency, multimodal support, and adaptability to user preferences. These insights contribute to the broader understanding of how LLM-based conversational agents can be effectively integrated into software development workflows while addressing adoption barriers and enhancing usability.