# Understanding the Human-LLM Dynamic: A Literature Survey of LLM Use in Programming Tasks

arXiv: https://arxiv.org/abs/2410.01026 · HTML: https://mlsanity.com/paper/2410.01026/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2024.

## Built on by (most cited first, 9 of 9)

- [Conversational AI as a Coding Assistant: Understanding Programmers' Interactions with and Expectations from Large Language Models for Coding](https://mlsanity.com/paper/2503.16508/) (2025)
- [Substance Beats Style: Why Beginning Students Fail to Code with LLMs](https://mlsanity.com/paper/2410.19792/) (2024)
- [Nexus: A Lightweight and Scalable Multi-Agent Framework for Complex Tasks Automation](https://mlsanity.com/paper/2502.19091/) (2025)
- [From Prompts to Propositions: A Logic-Based Lens on Student-LLM Interactions](https://mlsanity.com/paper/2504.18691/) (2025)
- [Curiosity by Design: An LLM-based Coding Assistant Asking Clarification Questions](https://mlsanity.com/paper/2507.21285/) (2025)
- [Representing LLMs in Prompt Semantic Task Space](https://mlsanity.com/paper/2509.22506/) (2025)
- [Personality Matters: User Traits Predict LLM Preferences in Multi-Turn Collaborative Tasks](https://mlsanity.com/paper/2508.21628/) (2025)
- [Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity](https://mlsanity.com/paper/2602.00723/) (2026)
- [Epitome: Pioneering an Experimental Platform for AI-Social Science Integration](https://mlsanity.com/paper/2507.01061/) (2025)

## Abstract (arXiv)

Large Language Models (LLMs) are transforming programming practices, offering significant capabilities for code generation activities. While researchers have explored the potential of LLMs in various domains, this paper focuses on their use in programming tasks, drawing insights from user studies that assess the impact of LLMs on programming tasks. We first examined the user interaction behaviors with LLMs observed in these studies, from the types of requests made to task completion strategies. Additionally, our analysis reveals both benefits and weaknesses of LLMs showing mixed effects on the human and task. Lastly, we looked into what factors from the human, LLM or the interaction of both, affect the human's enhancement as well as the task performance. Our findings highlight the variability in human-LLM interactions due to the non-deterministic nature of both parties (humans and LLMs), underscoring the need for a deeper understanding of these interaction patterns. We conclude by providing some practical suggestions for researchers as well as programmers.