How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
AI coding agents increasingly act directly within software environments, yet existing analyses of their failures rely on benchmark trajectories that miss how developers actually experience misalignment. We present an observational study of 20,574 coding-agent sessions from 1,639 repositories across IDE and CLI workflows. We operationalize misalignment as a breakdown made visible through developer pushback, and annotate each episode along four axes: form, cause, cost, and resolution. We identify seven recurring forms, spanning how agents read projects, interpret developer intent, follow rules, bound their actions, implement and execute code, and report progress. 90.50% of episodes impose effort and trust costs rather than irreversible system damage, yet 91.49% of visible resolutions still require explicit user correction. Misalignment patterns also differ across IDE and CLI settings, persist across adjacent sessions, and shift over time: while overall rates decline, constraint violations and inaccurate self-reporting grow in share. Our findings inform the design of training, evaluation, and interfaces for keeping coding agents aligned with real developer workflows.
Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 53 references that are in the map. First posted on arXiv 28 May 2026.
Builds on
12 of 53 references.
Closest ancestors · 7
Same direction, newest first
- 2026SWE-chat: Coding Agent Interactions From Real Users in the Wild22 Apr
- 2026ZORO: Active Rules for Reliable Vibe Coding17 Apr
- 2026Why Are AI Agent Involved Pull Requests (Fix-Related) Remain Unmerged? An Empirical Study29 Jan
- 2026Promises, Perils, and (Timely) Heuristics for Mining Coding Agent Activity26 Jan
- 2026Where Do AI Coding Agents Fail? An Empirical Study of Failed Agentic Pull Requests in GitHub21 Jan
- 2025Understanding Code Agent Behaviour: An Empirical Study of Success and Failure Trajectories31 Oct
- 2025Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows10 Jul
Other references · 5
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2024Continual Learning of Large Language Models: A Comprehensive Surveycited by 234
- 2026Programming by Chat: A Large-Scale Behavioral Analysis of 11,579 Real-World AI-Assisted IDE Sessionscited by 14
- 2025NaturalEdit: Code Modification through Direct Interaction with Adaptive Natural Language Representationcited by 2
Built on by
Citing papers per year: 2026: 14.
14 papers.
Sort:
By area:
- 2026Failure as a Process: An Anatomy of CLI Coding Agent Trajectoriescited by 13
- 2026Quantifying Overclaiming Propensity in Frontier LLM Agentscited by 4
- 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Modelcited by 3
- 2026Plans They Abandon, Reports They Author: The Narrative Layer of Autonomous Agentscited by 3
- 2026EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environmentscited by 2
- 2026How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Studycited by 2
- 2026From Conversation to Contribution: Characterizing Coding Agent in Open-Source Softwarecited by 0
- 2026When and How Context Rot Appears in Coding Agents: A White-Box Study of Agent Skills in Code Auditingcited by 0
- 2026Introducing Consort: A Spec-First Agent Framework for Enforced, Test-Driven Development on Live Database Branchescited by 0
- 2026Beyond Token Savings: A Systematic Study of Context Compression in LLM Agentscited by 0
- 2026Opera: A Verbal Critic Framework for Long-horizon Coding Agentscited by 0
- 2026The Agent Knew: Internal States Forecast Unauthorized Actions and Stop Them, and the Lies That Followcited by 0
- 2026What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports14 Oct 2026
- 2026Measuring and Reducing Unusable Replies of LLM Agents14 Oct 2026
- 2026The Agent Knew: Internal States Forecast Unauthorized Actions and Stop Them, and the Lies That Follow12 Oct 2026
- 2026Opera: A Verbal Critic Framework for Long-horizon Coding Agents27 Sep 2026
- 2026Beyond Token Savings: A Systematic Study of Context Compression in LLM Agents26 Sep 2026
- 2026Quantifying Overclaiming Propensity in Frontier LLM Agents17 Sep 2026
- 2026Plans They Abandon, Reports They Author: The Narrative Layer of Autonomous Agents10 Sep 2026
- 2026Introducing Consort: A Spec-First Agent Framework for Enforced, Test-Driven Development on Live Database Branches9 Sep 2026
- 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model14 Aug 2026
- 2026When and How Context Rot Appears in Coding Agents: A White-Box Study of Agent Skills in Code Auditing20 Jul 2026
- 2026How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Study12 Jul 2026
- 2026Failure as a Process: An Anatomy of CLI Coding Agent Trajectories10 Jul 2026
- 2026Failure as a Process: An Anatomy of CLI Coding Agent Trajectoriescited by 13
- 2026Quantifying Overclaiming Propensity in Frontier LLM Agentscited by 4
- 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Modelcited by 3
- 2026Plans They Abandon, Reports They Author: The Narrative Layer of Autonomous Agentscited by 3
- 2026How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Studycited by 2
- 2026From Conversation to Contribution: Characterizing Coding Agent in Open-Source Softwarecited by 0
- 2026When and How Context Rot Appears in Coding Agents: A White-Box Study of Agent Skills in Code Auditingcited by 0
- 2026Beyond Token Savings: A Systematic Study of Context Compression in LLM Agentscited by 0
- 2026Opera: A Verbal Critic Framework for Long-horizon Coding Agentscited by 0
- 2026The Agent Knew: Internal States Forecast Unauthorized Actions and Stop Them, and the Lies That Followcited by 0
- 2026Measuring and Reducing Unusable Replies of LLM Agentscited by 0
- 2026What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reportscited by 0
- 2026What Coding Agents Leave Unsaid: Measuring Task-State Gaps Between Execution and Reports14 Oct 2026
- 2026Measuring and Reducing Unusable Replies of LLM Agents14 Oct 2026
- 2026The Agent Knew: Internal States Forecast Unauthorized Actions and Stop Them, and the Lies That Follow12 Oct 2026
- 2026Opera: A Verbal Critic Framework for Long-horizon Coding Agents27 Sep 2026
- 2026Beyond Token Savings: A Systematic Study of Context Compression in LLM Agents26 Sep 2026
- 2026Quantifying Overclaiming Propensity in Frontier LLM Agents17 Sep 2026
- 2026Plans They Abandon, Reports They Author: The Narrative Layer of Autonomous Agents10 Sep 2026
- 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Model14 Aug 2026
- 2026When and How Context Rot Appears in Coding Agents: A White-Box Study of Agent Skills in Code Auditing20 Jul 2026
- 2026How Do Practitioners Build SE Agents? Insights from a Mixed-Methods Study12 Jul 2026
- 2026Failure as a Process: An Anatomy of CLI Coding Agent Trajectories10 Jul 2026
- 2026From Conversation to Contribution: Characterizing Coding Agent in Open-Source Software6 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026Beyond Resolution Rates: Behavioral Drivers of Coding Agent Success and Failure
- 2026Are We All Using Agents the Same Way? An Empirical Study of Core and Peripheral Developers Use of Coding Agents
- 2026SWE-Touch: Benchmarking Coding Agents When Users Touch the Code
- 2026SWE-MISBEHAVIOR: Benchmarking and Mitigating Boundary Violations in Software Coding Agents
- 2026ClayBuddy: A Framework, Evaluation, & Mitigation of Coding Agent Failures
- 2026Humans are Missing from AI Coding Agent Research
- 2026From Agent Behaviour to Agent-Friendly Documentation: An Empirical Study of How Coding Agents Discover, Read, and Write Technical Documentation
- 2026Investigating Autonomous Agent Contributions in the Wild: Activity Patterns and Code Change over Time