Humans are Missing from AI Coding Agent Research
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Recent progress in AI coding agent research has led to rapid improvements in agents' ability to autonomously perform complex software engineering tasks, from editing large codebases to executing long-horizon development workflows. As these systems make strides, however, the primary bottleneck to practical usefulness increasingly shifts away from pure task-solving capability, and toward challenges in how users communicate with, supervise, and trust agents. In this position paper, we argue for a reorientation from autonomous to human-centered coding agents: systems designed not only to complete tasks, but to collaborate effectively with people. We identify four core interaction-level dimensions that characterize the human-agent task-solving loop: task alignment, verifiability, steerability, and adaptability. Finally, we outline concrete research directions to advance these dimensions, including user-involved coding environments, comprehensive verification mechanisms, and principled measures of human-agent interaction quality.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 65 of its 111 references that are in the map. First posted on arXiv 4 Jul 2026.
Builds on
65 of 111 references.
Closest ancestors · 13
Same direction, newest first
- 2026Agentic Rubrics as Contextual Verifiers for SWE Agents7 Jan
- 2025Training Versatile Coding Agents in Synthetic Environments13 Dec
- 2025Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?17 Nov
- 2025How can we assess human-agent interactions? Case studies in software agent design10 Oct
- 2025Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows10 Jul
- 2025Understanding Software Engineering Agents: A Study of Thought-Action-Result Trajectories23 Jun
- 2025Why AI Agents Still Need You: Findings from Developer-Agent Collaborations in the Wild14 Jun
- 2025Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI26 May
- 2025R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents9 Apr
- 2024Training Software Engineering Agents and Verifiers with SWE-Gym30 Dec
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
- 2024SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents18 Jun
Other references · 52
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
- 2023Teaching Large Language Models to Self-Debugcited by 963
- 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
- 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
- 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
- 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
38 more
- 2022CodeT: Code Generation with Generated Testscited by 369
- 2024Agent Workflow Memorycited by 342
- 2020Evaluation of Text Generation: A Surveycited by 275
- 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 254
- 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
- 2022Grounded Copilot: How Programmers Interact with Code-Generating Modelscited by 216
- 2024SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?cited by 196
- 2025CollabLLM: From Passive Responders to Active Collaboratorscited by 117
- 2025SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Taskscited by 108
- 2025Inducing Programmatic Skills for Agentic Taskscited by 103
- 2023The Programmer's Assistant: Conversational Interaction with a Large Language Model for Software Developmentcited by 95
- 2022Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programmingcited by 91
- 2023A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challengescited by 75
- 2024TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Taskscited by 70
- 2024Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaborationcited by 60
- 2016A User Simulator for Task-Completion Dialoguescited by 56
- 2025Flipping the Dialogue: Training and Evaluating User Language Modelscited by 50
- 2024TestGenEval: A Real World Unit Test Generation and Test Completion Benchmarkcited by 49
- 2023"What It Wants Me To Say": Bridging the Abstraction Gap Between End-User Programmers and Code-Generating Large Language Modelscited by 48
- 2022Investigating Explainability of Generative AI for Code through Scenario-based Designcited by 47
- 2024LLM-Based Test-Driven Interactive Code Generation: User Study and Empirical Evaluationcited by 46
- 2024Need Help? Designing Proactive AI Assistants for Programmingcited by 42
- 2025Training Proactive and Personalized LLM Agentscited by 42
- 2025Magentic-UI: Towards Human-in-the-loop Agentic Systemscited by 41
- 2024Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Studycited by 40
- 2022What is it like to program with artificial intelligence?cited by 39
- 2023Investigating and Designing for Trust in AI-powered Code Generation Toolscited by 34
- 2025Future of Work with AI Agents: Auditing Automation and Augmentation Potential across the U.S. Workforcecited by 33
- 2024The RealHumanEval: Evaluating Large Language Models' Abilities to Support Programmerscited by 28
- 2025BugPilot: Complex Bug Generation for Efficient Learning of SWE Skillscited by 28
- 2023How Do Analysts Understand and Verify AI-Assisted Data Analyses?cited by 25
- 2025Copilot Arena: A Platform for Code LLM Evaluation in the Wildcited by 25
- 2025Coding Agents with Multimodal Browsing are Generalist Problem Solverscited by 24
- 2025How Do AI Agents Do Human Work? Comparing AI and Human Workflows Across Diverse Occupationscited by 17
- 2025How Developers Interact with AI: A Taxonomy of Human-AI Collaboration in Software Engineeringcited by 14
- 2025EDIT-Bench: Evaluating LLM Abilities to Perform Real-World Instructed Code Editscited by 13
- 2025CodingGenie: A Proactive LLM-Powered Programming Assistantcited by 10
- 2025AI & Human Co-Improvement for Safer Co-Superintelligencecited by 8
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026One Frozen Simulator Is Not Enough: Simulator Collapse in Multi-Agent RL12 Aug 2026
- 2026SWE-Touch: Benchmarking Coding Agents When Users Touch the Code3 Aug 2026
- 2026Interactive Task Alignment as a POMDP17 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2025From Correctness to Collaboration: Toward a Human-Centered Framework for Evaluating AI Agent Behavior in Software Engineering
- 2026SWE-Together: Evaluating Coding Agents in Interactive User Sessions
- 2025CentaurEval: Benchmarking Human-in-the-Loop Value in Agentic Coding
- 2026Humans Integrate, Agents Fix: How Agent-Authored Pull Requests Are Referenced in Practice
- 2026Human-AI Collaboration and the Transformation of Software Engineering Work
- 2026Are We All Using Agents the Same Way? An Empirical Study of Core and Peripheral Developers Use of Coding Agents
- 2026CooperBench: Why Coding Agents Cannot be Your Teammates Yet
- 2026Scaling Human-AI Coding Collaboration Requires a Governable Consensus Layer