WatchPoint: Executable User Feedback for Real-World Agentic Web Development
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
When a professional web developer's code fails a test, they do not simply re-read the stack trace. They open the application in a browser, click buttons, inspect computed styles, and run diagnostic commands to understand what went wrong. Existing feedback mechanisms for coding agents rely on screenshots, LLM-as-a-judge scoring, or natural-language corrections, but few interact with the live application the way a developer would. We introduce WatchPoint, a simulated-user system that mimics real developer behavior by generating and executing diagnostic scripts against the running application, producing structured observations that guide the coding model's retry. Unlike prior approaches that target single-file edits or evaluate using non-executable metrics, we operate on Web-Bench, a benchmark of 50 multi-file web projects comprising 1,000 sequentially dependent tasks, verified by deterministic end-to-end tests. WatchPoint recovers 57.6% of the tasks it diagnoses, and a controlled user study confirms the simulation's realism: human testers achieve a comparable recovery rate (54.5%), providing evidence that automated diagnostic scripts can substitute for interactive human testing on sequential web development tasks. We further identify a pattern of capability gaps that governs when simulated-user feedback is helpful and when it should be withheld.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 22 of its 25 references that are in the map. First posted on arXiv 10 Aug 2026.
Builds on
22 of 25 references.
Closest ancestors · 11
Same direction, newest first
- 2025ReLook: Vision-Grounded RL with a Multimodal LLM Critic for Agentic Web Coding13 Oct
- 2025When Agents go Astray: Course-Correcting SWE Agents with PRMs2 Sep
- 2025Web-Bench: A LLM Code Benchmark Based on Web Standards and Frameworks12 May
- 2025Why Do Multi-Agent LLM Systems Fail?17 Mar
- 2024Human-In-the-Loop Software Development Agents19 Nov
- 2024SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement26 Oct
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
- 2024Agentless: Demystifying LLM-based Software Engineering Agents1 Jul
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May
- 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Ahead7 Apr
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug
Other references · 11
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2021Program Synthesis with Large Language Modelscited by 3,509
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2024HAICOSYSTEM: An Ecosystem for Sandboxing Safety Risks in Human-AI Interactionscited by 42
- 2026Lost in Simulation: LLM-Simulated Users are Unreliable Proxies for Human Users in Agentic Evaluationscited by 36
- 2025ArtifactsBench: Bridging the Visual-Interactive Gap in LLM Code Generation Evaluationcited by 35
- 2025SWE-Bench++: A Framework for the Scalable Generation of Software Engineering Benchmarks from Open-Source Repositoriescited by 27
- 2025FronTalk: Benchmarking Front-End Development as Conversational Code Generation with Multi-Modal Feedbackcited by 14
- 2025WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learningcited by 11
- 2025FeedbackEval: A Benchmark for Evaluating Large Language Models in Feedback-Driven Code Repair Taskscited by 8
- 2025InspectCoder: Dynamic Analysis-Enabled Self Repair through interactive LLM-Debugger Collaborationcited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ProgramDistill: From Interactive Web Apps to Verifiable Reference-Guided SWE Tasks
- 2026Asuka-Bench: Benchmarking Code Agents on Underspecified User Intent and Multi-Round Refinement
- 2026From Runnable to Shippable: Multi-Agent Test-Driven Development for Generating Full-Stack Web Applications from Requirements
- 2026AppLooper: An Agentic Application Engineering Loop for Accountable Release with Virtual-User Feedback
- 2026Framework and Benchmark for Code-Driven Agentic Testing in Web Development
- 2026SWE-Touch: Benchmarking Coding Agents When Users Touch the Code
- 2026Agentic Code Review in the Terminal: A Trajectory-Level Analysis of Behavior, Cost, and Human-Alignment
- 2025WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch