mlsanityData to 14 Oct 2026

How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions

arXiv 2605.29442 ↗First posted 28 May 2026cs.SE cs.AI cs.HCcited by 14

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

AI coding agents increasingly act directly within software environments, yet existing analyses of their failures rely on benchmark trajectories that miss how developers actually experience misalignment. We present an observational study of 20,574 coding-agent sessions from 1,639 repositories across IDE and CLI workflows. We operationalize misalignment as a breakdown made visible through developer pushback, and annotate each episode along four axes: form, cause, cost, and resolution. We identify seven recurring forms, spanning how agents read projects, interpret developer intent, follow rules, bound their actions, implement and execute code, and report progress. 90.50% of episodes impose effort and trust costs rather than irreversible system damage, yet 91.49% of visible resolutions still require explicit user correction. Misalignment patterns also differ across IDE and CLI settings, persist across adjacent sessions, and shift over time: while overall rates decline, constraint violations and inaccurate self-reporting grow in share. Our findings inform the design of training, evaluation, and interfaces for keeping coding agents aligned with real developer workflows.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 53 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

12 of 53 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026SWE-chat: Coding Agent Interactions From Real Users in the Wild22 Apr
  2. 2026ZORO: Active Rules for Reliable Vibe Coding17 Apr
  3. 2026Why Are AI Agent Involved Pull Requests (Fix-Related) Remain Unmerged? An Empirical Study29 Jan
  4. 2026Promises, Perils, and (Timely) Heuristics for Mining Coding Agent Activity26 Jan
  5. 2026Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub21 Jan
  6. 2025Understanding Code Agent Behaviour: An Empirical Study of Success and Failure Trajectories31 Oct
  7. 2025Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows10 Jul

Other references · 5

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  3. 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
  4. 2026Programming by Chat: A Large-Scale Behavioral Analysis of 11,579 Real-World AI-Assisted IDE Sessionscited by 14
  5. 2025NaturalEdit: Code Modification through Direct Interaction with Adaptive Natural Language Representationcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Resolution Rates: Behavioral Drivers of Coding Agent Success and Failure
  2. 2026Are We All Using Agents the Same Way? An Empirical Study of Core and Peripheral Developers Use of Coding Agents
  3. 2026SWE-Touch: Benchmarking Coding Agents When Users Touch the Code
  4. 2026SWE-MISBEHAVIOR: Benchmarking and Mitigating Boundary Violations in Software Coding Agents
  5. 2026ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failures
  6. 2026Humans are Missing from AI Coding Agent Research
  7. 2026From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation
  8. 2026Investigating Autonomous Agent Contributions in the Wild: Activity Patterns and Code Change over Time