mlsanityData to 7 Oct 2026

Humans are Missing from AI Coding Agent Research

arXiv 2608.12355 ↗First posted 4 Jul 2026cs.HC cs.AI cs.SEcited by 3

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Recent progress in AI coding agent research has led to rapid improvements in agents' ability to autonomously perform complex software engineering tasks, from editing large codebases to executing long-horizon development workflows. As these systems make strides, however, the primary bottleneck to practical usefulness increasingly shifts away from pure task-solving capability, and toward challenges in how users communicate with, supervise, and trust agents. In this position paper, we argue for a reorientation from autonomous to human-centered coding agents: systems designed not only to complete tasks, but to collaborate effectively with people. We identify four core interaction-level dimensions that characterize the human-agent task-solving loop: task alignment, verifiability, steerability, and adaptability. Finally, we outline concrete research directions to advance these dimensions, including user-involved coding environments, comprehensive verification mechanisms, and principled measures of human-agent interaction quality.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 65 of its 111 references that are in the map. First posted on arXiv 4 Jul 2026.

Builds on

65 of 111 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026Agentic Rubrics as Contextual Verifiers for SWE Agents7 Jan
  2. 2025Training Versatile Coding Agents in Synthetic Environments13 Dec
  3. 2025Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?17 Nov
  4. 2025How can we assess human-agent interactions? Case studies in software agent design10 Oct
  5. 2025Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows10 Jul
  6. 2025Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories23 Jun
  7. 2025Why AI Agents Still Need You: Findings from Developer-Agent Collaborations in the Wild14 Jun
  8. 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI26 May
  9. 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents9 Apr
  10. 2024Training Software Engineering Agents and Verifiers with SWE-Gym30 Dec
  11. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
  12. 2024SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents18 Jun
1 more
  1. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May

Other references · 52

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  5. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  6. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  7. 2023Teaching Large Language Models to Self-Debugcited by 963
  8. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  9. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  10. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  11. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  12. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
38 more
  1. 2022CodeT: Code Generation with Generated Testscited by 369
  2. 2024Agent Workflow Memorycited by 342
  3. 2020Evaluation of Text Generation: A Surveycited by 275
  4. 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 254
  5. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
  6. 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 216
  7. 2024SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?cited by 196
  8. 2025CollabLLM: From Passive Responders to Active Collaboratorscited by 117
  9. 2025SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Taskscited by 108
  10. 2025Inducing Programmatic Skills for Agentic Taskscited by 103
  11. 2023The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Developmentcited by 95
  12. 2022Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programmingcited by 91
  13. 2023A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challengescited by 75
  14. 2024TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Taskscited by 70
  15. 2024Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaborationcited by 60
  16. 2016A User Simulator for Task-Completion Dialoguescited by 56
  17. 2025Flipping the Dialogue: Training and Evaluating User Language Modelscited by 50
  18. 2024TestGenEval: A Real World Unit Test Generation and Test Completion Benchmarkcited by 49
  19. 2023"What It Wants Me To Say": Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Modelscited by 48
  20. 2022Investigating Explainability of Generative AI for Code through Scenario-based Designcited by 47
  21. 2024LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluationcited by 46
  22. 2024Need Help? Designing Proactive AI Assistants for Programmingcited by 42
  23. 2025Training Proactive and Personalized LLM Agentscited by 42
  24. 2025Magentic-UI: Towards Human-in-the-loop Agentic Systemscited by 41
  25. 2024Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Studycited by 40
  26. 2022What is it like to program with artificial intelligence?cited by 39
  27. 2023Investigating and Designing for Trust in AI-powered Code Generation Toolscited by 34
  28. 2025Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforcecited by 33
  29. 2024The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmerscited by 28
  30. 2025BugPilot: Complex Bug Generation for Efficient Learning of SWE Skillscited by 28
  31. 2023How Do Analysts Understand and Verify AI-Assisted Data Analyses?cited by 25
  32. 2025Copilot Arena: A Platform for Code LLM Evaluation in the Wildcited by 25
  33. 2025Coding Agents with Multimodal Browsing are Generalist Problem Solverscited by 24
  34. 2025How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupationscited by 17
  35. 2025How Developers Interact with AI: A Taxonomy of Human-AI Collaboration in Software Engineeringcited by 14
  36. 2025EDIT-Bench: Evaluating LLM Abilities to Perform Real-World Instructed Code Editscited by 13
  37. 2025CodingGenie: A Proactive LLM-Powered Programming Assistantcited by 10
  38. 2025AI & Human Co-Improvement for Safer Co-Superintelligencecited by 8

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025From Correctness to Collaboration: Toward a Human-Centered Framework for Evaluating AI Agent Behavior in Software Engineering
  2. 2026SWE-Together: Evaluating Coding Agents in Interactive User Sessions
  3. 2025CentaurEval: Benchmarking Human-in-the-Loop Value in Agentic Coding
  4. 2026Humans Integrate, Agents Fix: How Agent-Authored Pull Requests Are Referenced in Practice
  5. 2026Human-AI Collaboration and the Transformation of Software Engineering Work
  6. 2026Are We All Using Agents the Same Way? An Empirical Study of Core and Peripheral Developers Use of Coding Agents
  7. 2026CooperBench: Why Coding Agents Cannot be Your Teammates Yet
  8. 2026Scaling Human-AI Coding Collaboration Requires a Governable Consensus Layer