# Generation of Web Apps with Agentic IDEs: An Empirical Assessment

arXiv: https://arxiv.org/abs/2608.20903 · HTML: https://mlsanity.com/paper/2608.20903/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 32 references that are in the map.
- First posted on arXiv 21 Aug 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 21 of 21)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Program Synthesis with Large Language Models](https://mlsanity.com/paper/2108.07732/) (2021)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation](https://mlsanity.com/paper/2303.12570/) (2023)
- [CodePlan: Repository-level Coding using LLMs and Planning](https://mlsanity.com/paper/2309.12499/) (2023)
- [Productivity Assessment of Neural Code Completion](https://mlsanity.com/paper/2205.06537/) (2022)
- [Using AI-Based Coding Assistants in Practice: State of Affairs, Perceptions, and Ways Forward](https://mlsanity.com/paper/2406.07765/) (2024)
- [HITS: High-coverage LLM-based Unit Test Generation via Method Slicing](https://mlsanity.com/paper/2408.11324/) (2024)
- [Assistance or Disruption? Exploring and Evaluating the Design and Trade-offs of Proactive AI Programming Support](https://mlsanity.com/paper/2502.18658/) (2025)
- [Code with Me or for Me? How Increasing AI Automation Transforms Developer Workflows](https://mlsanity.com/paper/2507.08149/) (2025)
- [On the Impacts of Contexts on Repository-Level Code Generation](https://mlsanity.com/paper/2406.11927/) (2024)
- [DependEval: Benchmarking LLMs for Repository Dependency Understanding](https://mlsanity.com/paper/2503.06689/) (2025)
- [Human-Written vs. AI-Generated Code: A Large-Scale Study of Defects, Vulnerabilities, and Complexity](https://mlsanity.com/paper/2508.21634/) (2025)
- [LLM Test Generation via Iterative Hybrid Program Analysis](https://mlsanity.com/paper/2503.13580/) (2025)
- [Studying How Configurations Impact Code Generation in LLMs: the Case of ChatGPT](https://mlsanity.com/paper/2502.17450/) (2025)
- [Analyzing Prompt Influence on Automated Method Generation: An Empirical Study with Copilot](https://mlsanity.com/paper/2402.08430/) (2024)
- [Generating Java Methods: An Empirical Assessment of Four AI-Based Code Assistants](https://mlsanity.com/paper/2402.08431/) (2024)
- [Assessing AI-Based Code Assistants in Method Generation Tasks](https://mlsanity.com/paper/2402.09022/) (2024)
- [AI Code in the Wild: Measuring Security Risks and Ecosystem Shifts of AI-Generated Code in Modern Software](https://mlsanity.com/paper/2512.18567/) (2025)

## Abstract (arXiv)

Agentic IDEs are among the most significant innovations in software engineering, aiming to accelerate application development through LLM-based agents that can assist developers during development. However, their evaluation in end-to-end development tasks involving the generation of complete applications remains limited. To fill this gap, we propose a rigorous comparative analysis of three popular agentic IDEs (Copilot, Cursor, and Windsurf) in the generation of five full-stack Web applications from scratch. Results show high maturity in the generation of established patterns, such as CRUD operations and authentication features. In contrast, the generation of less common distributed architectures, such as a task queue architecture, produces significantly more errors. Overall, results show that Agentic IDEs cannot replace developers but shift their role toward building software by orchestrating LLM-based agents through natural-language instructions and iterative refinement. Yet, each agentic IDE shows its peculiarities, although differences are narrow.