mlsanityData to 7 Oct 2026

SWE-Touch: Benchmarking Coding Agents When Users Touch the Code

arXiv 2608.02499 ↗First posted 3 Aug 2026cs.SE cs.AI cs.CLcited by 2

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Real-world software development requires coding agents to operate in shared workspaces where users may inspect and modify code during an ongoing task, yet existing repository-level benchmarks typically evaluate agents working alone or restrict user participation to messages. This leads us to ask: how do coding agents understand and respond to code changes in a shared workspace? We introduce SWE-Touch, a framework that stress-tests this setting through validated Counter-Edits: plausible edits to task-relevant code that conflict with task completion. SWE-Touch mines task-critical regions from multiple repair trajectories, uses a separate User Patch Generator to construct the edits, and injects them with contextual user messages when agents reach the relevant code. We evaluate nine coding models on SWE-bench Verified, with additional experiments on longer-horizon tasks from SWE-Bench Pro and DeepSWE. Counter-Edit lowers average resolve rate by 7.7 percentage points on SWE-bench Verified, with degradation also persisting on both longer-horizon benchmarks. Trajectory analysis links these failures to limited awareness of the evolving workspace: agents may retain conflicting code or replace it without sufficiently re-inspecting the repository and validating the revised code with targeted tests. These findings show that strong autonomous performance does not yet ensure the state awareness and adaptive behavior needed for shared-workspace collaboration, and point to detecting workspace changes, reconciling conflicting edits with the task, and verifying the affected behavior as key capabilities for future optimization.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 26 of its 52 references that are in the map. First posted on arXiv 3 Aug 2026.

Builds on

26 of 52 references.

Closest ancestors · 12

Same direction, newest first

  1. 2026ICAE-Bench: Evaluating Coding Agents as Interactive Project Builders23 Jul
  2. 2026Humans are Missing from AI Coding Agent Research4 Jul
  3. 2026SWE-INTERACT: Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding Sessions29 Jun
  4. 2026Dialogue SWE-Bench: A Benchmark for Dialogue-Driven Coding Agents12 Jun
  5. 2026EvoCode-Bench: Evaluating Coding Agents in Multi-Turn Iterative Interactions22 May
  6. 2026Hedwig: Dynamic Autonomy for Coding Agents Under Local Oversight12 May
  7. 2026SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks25 Mar
  8. 2025SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios20 Dec
  9. 2025How can we assess human-agent interactions? Case studies in software agent design10 Oct
  10. 2025SWE-smith: Scaling Data for Software Engineering Agents30 Apr
  11. 2025SyncMind: Measuring Agent Out-of-Sync Recovery in Collaborative Software Engineering10 Feb
  12. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May

Other references · 14

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  2. 2024GPT-4o System Cardcited by 4,467
  3. 2021Program Synthesis with Large Language Modelscited by 3,509
  4. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  5. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 558
  6. 2024BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructionscited by 527
  7. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 515
  8. 2023A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challengescited by 75
  9. 2024DA-Code: Agent Data Science Code Generation Benchmark for Large Language Modelscited by 57
  10. 2025Non-Collaborative User Simulators for Tool Agentscited by 17
  11. 2026HiL-Bench (Human-in-Loop Benchmark): Do Agents Know When to Ask for Help?cited by 16
  12. 2026Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agentscited by 15
2 more
  1. 2025EDIT-Bench: Evaluating LLM Abilities to Perform Real-World Instructed Code Editscited by 13
  2. 2024DEMO: Reframing Dialogue Interaction with Fine-grained Element Modelingcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026SWE-Together: Evaluating Coding Agents in Interactive User Sessions
  2. 2026SWE-Explore: Benchmarking How Coding Agents Explore Repositories
  3. 2026How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions
  4. 2026Coherence Collapse: Diagnosing Why Code Agents Fail After Reaching the Right Code
  5. 2026SWE-chat: Coding Agent Interactions From Real Users in the Wild
  6. 2026SWE Atlas: Benchmarking Coding Agents Beyond Issue Resolution
  7. 2026SWE-Cycle: Benchmarking Code Agents across the Complete Issue Resolution Cycle
  8. 2025TOM-SWE: User Mental Modeling For Software Engineering Agents